The Role and Difficulty of Skill-Based Matchmaking

Activision’s findings about skill-based matchmaking (SBMM) are an important read about the challenges and misconceptions of creating a competitive and friendly online atmosphere. But it also drives home one of the hardest aspects of these games and how it relates to matchmaking and balancing a game: what does skill mean in a game?

A winner is...you?

When we discuss single player or linear experiences, it's far easier to define a game's skill curve: if the player beats Level 1, they can now do Level 2. Every reflex-driven game is about constantly testing the player's mastery and understanding of a game as they make progress through the experience.

However, progression looks different in multiplayer games. Let's say I'm playing against a team of people who have no clue what they are doing. If I walk away with a kill/death ratio (K/D) of 15/2, then I must be some kind of pro gamer...right? But on the very next match, I'm now up against a team who knows how to play and my K/D ratio completely changes to 2/15.

Whenever a human player goes up against another human player, the number of factors that affect a match is exponentially greater than in a single player experience. Even weapons in a multiplayer context add much more complexity to the equation. Now, it's not just a question of how powerful a particular weapon is, it's also a question of each human player's individual skill with each specific weapon (and how these factors compare in a multiplayer encounter). When teams are added to the mix, the ability for a developer to predict player skill level becomes almost impossible.

Defining "skill" through measurable metrics

Anyone who has played a team-based game knows that hell is truly other people. Being the best or worst player on a team can feel like torture. You might either feel like you're carrying the entire team or letting them down. But this also creates a problem in terms of measuring skill, especially in relation to teamwork and especially if your game features mechanics or elements that aren't easily quantified. Measurable metrics of wins and losses, kills and deaths, hits or misses, and so on, are easily tracked. However, there are "fuzzy" factors that lead to these results which are far harder to gauge.

This problem I'm describing is a longstanding issue with Dead by Daylight, which has the added challenge of an asymmetrical encounter design. The game's matchmaking is based on determining "good play" - and again, "good" is often synonymous with "winning". To win as the killer, you must hook the other team enough times to defeat them. Winning as a survivor means escaping. But here's the problem: there is far more happening in a match that contributes to these end states but that isn't factored into the assessment of win/loss.

Let's say a killer fails to defeat the survivors but manages to get the entire team on the "death hook" state and holds out for a long time. Were they a good or bad killer? And then, in the case of survivors, let's say someone is the first to be killed - but they were constantly juking the killer and taking hits for the entire team. Were they bad at the game? Both sides are awarded points based on various factors of 'smart play', but the ability to acquire these points is, again, going to be determined by the skill of the killer and the survivors.

To add even more confusion to the mix, let's introduce the concept of "ELO hell" (ELO is a rating system that measures the relative skill levels of players in zero-sum games). One of the ways this articulates in real life scenarios is to identify cases where a player is sitting in the wrong matchmaking bracket due to their team. If I'm a really good player and I'm on a team of newcomers, I could end up being completely shut down by the opposing team. Does that mean I'm a horrible player? Conversely, if I'm the standout player among a team of lesser-skilled people, does that mean I'm a brilliant player?

What does all this mean? Understanding what skill means in your game involves figuring out how a team/player can achieve particular win states, but also, how they contribute during the game. Let's say your game only tracks kills as the sole determination of skill. But if I'm playing a support role - maybe a tank or healer - and I keep my entire team alive but never score a kill, am I a bad player? Am I omitted from win states or points/acknowledgement because I haven't satisfied a specific narrow win condition?

Games that reward "good play" or "skilled play" need to be cautious about precisely how these things are measured and determined. This isn't just because players will miss out on advancement, it's also because highly limited measures of success may cause the player base to gravitate to highly restricted ways of playing the game. In other words, limited success metrics can incentivize limited tactics/ways to play.

Unlocking the meta

Speaking of incentives, it's worth noting that whatever is seen as the "best" way to play - which may become the most common way to play - tends to most negatively impact players at both ends of the spectrum (e.g. casual players on one side and hardcore players on the other). If everyone knows that Gun X, Perk Y, and Character Z are the best combinations, then why should anyone who wants to move up in the ranking play anything else? This leads to a kind of toxic behavior in these games, where some players bully others if they aren't playing what's currently meta. From MOBAs, to shooters, to MMORPGs, people want to win, as there's certainly no reward for losing a match.

This often leads to poor sportsmanship and people using rankings as a cudgel against other players. It is the common refrain: "It's not my fault we lost, I had the best score on my team." The problem is that you can play "correctly", do everything well, and your team still loses; the opposite is also true. When skills and progression in these complex settings are whittled down to only 2 or 3 metrics, the impact can be that players don't see or care about the many ways that you might be contributing to team success (keeping everyone alive, delaying/holding back the other team, etc...) - if you're the lowest on the scoreboard, then it must have been all your fault. Many games have tried to combat the toxicity present in multiplayer experiences by incorporating stricter and more powerful tools for reporting bad behavior.

That last point also surfaces something important about how multiplayer and skill-based games have transformed over the years.

The limits of skill and matchmaking

There's one more fallacy that I want to touch on. Everything here is based on the idea of designing a competitive/esport-friendly experience, but this is only one part of the broader equation. Even if you do find the magic ratio and craft a perfectly-balanced competitive experience, you still have all your work ahead of you of you: that is, crafting an enjoyable experience that people actually want to play.

You can give me all the best skill-based matchmaking in the world, but that won't mean I automatically enjoy the fighting, battle royale, multiplayer shooter, etc... experience you're building. Multiplayer design has become more and more homogenized over the years, with developers chasing both the live service model (the idea of making a "daily game" or a "lifestyle game" that players regularly dive into) while also trying to appeal to the esports crowd. Games designed to meet these two needs can sometimes exclude players who want to dip in once in a while, or play casually with friends, or casually with random people. It's often the case that you must play these games every day or you're going to fall behind in the meta, your ranking, or whatever daily rewards are included.

It feels like forever since we've seen someone create a unique multiplayer experience that genuinely hasn't been seen before. When everything is filtered through the idea of a daily play, team-based/hero shooter, or fighting game, there are so few ways you can stand out from everyone else.

For readers: What are your thoughts on skill-based matchmaking? And can multiplayer design continue to grow in new/novel directions?