Report: Exploring Weekend Events

Editorial Disclaimer

This post was originally published by Spencatro on 08/27/2018. Spencatro has since gone on to work at Wizards of the Coast (Dec 2018). MTGATracker remains a 3rd-party project that is not affiliated with Wizards of the Coast, and is published pursuant to the Wizards of the Coast Fan-Content Policy. The views and opinions expressed in this post are strictly those of the author, and do not reflect the official position, policy, views, or opinions of Wizards of the Coast. No authors were compensated by any parties for the authorship of this post.

Wizards of the Coast is pushing the boundaries of what MTG can be with the new weekend series events in Magic: The Gathering Arena. This past weekend, players got to see what MTG is like with a few core rules modified: namely, players started each game with 9 cards (instead of 7), could play 2 lands each turn (instead of 1), and had no maximum hand size. While these new rules are a far cry from the crazy stuff we get to see in UN-sets, it’s a signal that WotC are open to experimenting with the rules- in a style not dissimilar from Hearthstone’s “Tavern Brawl” mode.

While players are excited to get to try new types of play too young to have defined metas, they’re less excited about an integral part of the new modes: their rewards structures. Like many of the topics we discuss on this blog, complaints about prize support aren’t unique to MTGA, but paying 500 gold for a chance at either a single pack or a single uncommon card–which could end up being a duplicate card, turned in for 0.3% vault progress–has players questioning if they’ll be able to afford to give these events the shot they deserve.

Will It Blend?

It’s clear that many players don’t like the reward structure for the new events, but did this affect how much they took part in these events? Let’s look at MTGATracker data to find out!

UPDATE: 8/30/18 ~11AM PST

Chris Clay, Game Director of MTGA at Wizards of the Coast, has published an update on Quick Exploration-style events on the official MTGA forums. We’ve included an excerpt from it below:

The reward structure around the Exploration event has been a hot button issue, and as promised we wanted to share some of what we learned over the weekend. Looking at player participation, we’re able to identify a few interesting points:

Exploration was the most played, non-ladder, constructed event over the weekend. As a baseline for comparison, we saw more unique players participate in Exploration than in Singleton the previous week. Of those players, more participated in Exploration at least twice than Singleton twice. Initial analysis points towards the event being more popular with players who typically play for an hour or less a day, than with those that typically play more. Exploration was not, however, as popular as Singleton was in its first weekend. What this tells us is that there were enough people happy to play in the event, it was played all weekend long, and it appealed to those players who typically play for shorter periods of time. But, it also tells us players weren’t as excited for Exploration as they were initially for Singleton, and it wasn’t something that players went out of their way to participate in.

As we’re sure you’ll guess, this is likely because of the prize structure. … This bring us to our next weekend event, Pauper. We’re going to be testing a new event and prize structure …

Pauper Reward Structure

Entry Fee: 500 gold or 100 gems

Match Structure: Single Game

Event Ends After: 5 wins or 2 losses, whichever comes first.

Rewards:

• 5 wins: 600 gold + 1 rare/mythic ICR + 1 uncommon ICR
• 4 wins: 500 gold + 1 rare/mythic ICR + 1 uncommon ICR
• 3 wins: 400 gold + 1 rare/mythic ICR + 1 uncommon ICR
• 2 wins: 300 gold + 1 rare/mythic ICR + 1 uncommon ICR
• 1 win: 200 gold + 1 rare/mythic ICR + 1 uncommon ICR
• 0 wins: 100 gold + 1 rare/mythic ICR + 1 uncommon ICR
  • Note that uncommon ICRs have a 15% probability to upgrade to rare or mythic

At MTGATracker, we view this change extremely favorably, and this writer would like to personally thank Chris and the rest of the MTGA team for taking players’ feedback to heart (and implementing a change so quickly)! Whether this post helped influence this decision or not, we view this change as a big win for both players and Wizards of the Coast alike.

That said, astute readers will notice that this statement does seem to fly in the face of the data published in this report. For example, this report does not reflect the statement that “Exploration was the most played, non-ladder, constructed event over the weekend.”

We believe in the integrity of MTGATracker data; we also believe in the integrity of Wizards of the Coasts’ data. And most of all, we have no reason to believe that Chris or Wizards have reported inaccurate information. To us, this means that (as some readers have pointed out), MTGATracker data may be not entirely representative of all MTGA players. This is a hard pill to swallow, but not entirely unexpected.

We’ll continue writing reports and analyzing the data we have access to because, let’s face it, it’s fun as hell (and pretty dang cool)! However, readers should be aware that MTGATracker data comes from a self-selecting sample of users, which means it likely will not represent every player. Only Wizards of the Coast has access to their entire dataset, so if / when WotC publishes data that conflicts with MTGATracker reports, it should be understood that WotC’s data is more comprehensive and more accurately represents the MTGA playerbase as a whole.

Will It Blend? (cont.)

Before we go too much further, just one quick plug: MTGATracker is on Patreon! MTGATracker is completely free and open source, and WTTT is presented without advertisements. If you enjoy these reports, please consider contributing to MTGATracker to let us know that we should keep doing what we’re doing, and also help keep us ad-free!

The first MTGA game in the Quick Exploration event was recorded with MTGATracker on 8/24/18 at 8:28AM PST, and the last on 8/27/18 at 9:43AM PST. For the purposes of this report, we will generally be looking at the 5,003 games that were recorded within this time frame.

We can see that during this timeframe, 355 games (or, roughly 7%) were in the Quick Exploration event.

Relative popularity of events is worth noting itself, but it might not tell the whole story. Instead of looking at the average popularity of the entire period altogether, we also built a graph showing a “sliding average” popularity, which can show shifts in event popularity over time. (You can find the code that generated this analysis, along with anonymized data sets, in the MTGATracker labs repo.)

This graph gets a bit cramped towards the bottom. We don’t really care about the ladder (we get it, it’s super popular), so let’s zoom in to get a better look at the plethora of less popular event types below it:

Despite a few peaks and dips, Quick Exploration generally stayed stable between 5% and 8% of all games played. Note that the right side of the graph starts to get a little erratic; this is due to the “sliding window” approach running out of extra data towards the end of our dataset. When this happens, the window gets compressed and is therefore more susceptible to short-term trends. For example, this graph shows what this data would look like without the “smoothing” effect of the sliding windows. Chaos! We chose to include the data on the right side of the graph anyways- as a result, we can see Quick Exploration’s decline as it left the rotation of game modes.

This data is pretty to look at, but the story it tells isn’t exactly clear. One piece of data we can use to interpret these values is the 3% cutoff WotC used to justify the plan to remove Competitive Constructed from the game. Clearly, for one weekend, Quick Exploration at least meets this bar.

To get some more context, let’s try comparing it to something else: the event it replaced, Quick Singleton! While Quick Singleton is probably the best format to compare Quick Exploration to, it isn’t perfect. First, Quick Singleton lasted for much, much longer than Quick Exploration, which means that we have a much larger dataset to analyze. Furthermore, Quick Singleton events can last between 3 and 9 games, while Quick Exploration events are much shorter at 2-3 games. Finally, Quick Exploration might see some artificial inflation due to the fact that it was a time-limited weekend-only event.

Enough caveats, let’s see the numbers!

This data is a perfect candidate to be drawn with sliding windows for a couple of reasons. First (since this is a much larger period of data), Quick Draft is separated by format, which makes it appear as if Quick Singleton is the third most popular mode. However, if you group all QuickDraft segments together (DOM, AKH, M19), QuickDraft goes back to second place at about 17%. Next, as we’re about to see, Quick Singleton doesn’t appear to have done better than Quick Exploration- but when we look at the sliding windows, Quick Singleton actually peaked much higher (around 13.8%, compared to Quick Exploration’s 8.3%) and maintained its plateau for a much longer time.

There are a few other fun things to note about this graph. Using the sliding windows method, we can actually watch QuickDraft events rotate, and we can see a more accurate representation of their “live” popularity. For example, while it was in rotation, AKH Quick Draft peaked at much higher than the 5% it’s represented by in the pie chart, but was still notably less popular with players than DOM Quick Draft.

“Explore once, shame on me. Explore twice… you can’t- you can’t get explore’d again.”

Next, let’s find out how many times each player participated in the Quick Exploration event. To do this, we split our query up by player as well. Note that since MTGATracker doesn’t (yet) track distinct events^† and that a single event could be comprised of 2 or 3 games, we can’t do simple division to find out how many times each player entered the event. Instead, we’ll have to iterate over each game and check the player’s win/loss record to decide if they had to re-enter the event to play the next game^††.

^†Want to help make MTGATracker better? Open a PR for this issue!
^††For example, if a user's game history looks like: WWLWLLLLWW, we can determine they enetered the event four times: WW / LWL / LL / LWW

Once we know how many times each player entered the event, we can separate them into “buckets” and measure the size of each bucket.

This one almost isn’t even worth graphing. While there are a few who can’t seem to get enough, a clear majority of players aren’t coming back to Quick Exploration. There are three distinct populations here: those who never played Quick Explore, those who entered QE exactly once, and those who entered it multiple times.

Let’s see what Quick Singleton looked like:

This one was surprising to us. This writer expected Quick Singleton to be significantly more popular than Quick Exploration (and frankly, I’m a little disappointed that it isn’t moreso). On the other hand, the tail here is significantly longer, and more players came back for more than one event than those who quit after one. It’s also worth noting that completing a Quick Singleton event can in some cases be twice as many games as a Quick Exploration event, and therefore twice as much of a commitment. All of that said, there are still a sizeable number of one-and-done players, even in Quick Singleton.

The Bottom Line?

While Quick Exploration certainly didn’t do as well as Quick Singleton, it also wasn’t exactly a total flop, either. This writer still hopes that Wizards of the Coast will consider the ways in which their prize structure has affected player turnout rather than blaming the format outright. Quick Exploration is a really cool idea that I personally would have loved to have put more gold and time into! I’m excited about the introduction of unsolved formats, and I’m glad that WotC are open to experimentation on core MTG rules. Unfortunately, I was one of the “single entry” players for QE this past weekend, and–while I can’t speak for all of us–I personally chose to abstain from participating in additional rounds of QE in order to encourage the Arena team to rework this format’s prize structure.

Addendum: Collection Methodology

All of the data shown in this report is real data aggregated from MTGATracker users. When MTGATracker observes the conclusion of a game in MTGA, it uploads some data to MTGATracker servers, including the following (which has been truncated for simplicity):

{
  "players": [{
      "name": "Spencatro",
      // ...
    },
    {
      "name": "Opponent's MTGA Username",
      // ...
    }
  ],
  "winner": "Spencatro",
  "eventID": "QuickExploration_08242018",
  // ...
}

The following are the names MTGATracker reads from the MTGA log file (and is included in the data set in the labs repo), and the human-friendly names we’ve given them in this document:

You can use the Log Names to search your own local MTGA log for the raw version of the data MTGATracker interprets from hundreds of MTGATracker users to generate this report.

Thanks for reading! We’ll be back with another meta report shortly before the release of Guilds of Ravnica- and we may also have a few surprise posts between now and then. If you have an idea that you’d like us to write about (or if you’re interested in writing a piece yourself), let us know in Discord! If you’ve enjoyed reading this report, or want to directly support the development of MTGATracker, please consider contributing to MTGATracker on Patreon! Otherwise, we’ll see you at the next report!

Keep trackin, trackers

Errata, Corrections, and Changes

• An earlier version of this post referred to all event names by their WotC log “human-unfriendly” names (e.g. “Quick Play” was referred to as “ConstructedRanked1”). We replaced all instances with human-friendly names, and included a table mapping each human-friendly name to it’s WotC log counterpart in the Addendum section

• An earlier version of this post sometimes ambiguously referred to Quick Exploration as “Quick Explore.” These have all be changed to Quick Exploration.

• Added 8/26/18 update linking to Chris Clay’s forum post regarding Quick Exploration rewards