Physical Vocabulary for Digital Worlds

Nineteen Eighty-Four is my favourite novel; I must have read it half a dozen times. There are many reasons why I believe it to be a work of literary genius, but recently I’ve been thinking about one specific aspect of it that has a very clear analogue to video games: vocabulary. More specifically, I’m referring to the idea that applying deliberate constraints to vocabulary can directly restrict the concepts that one can think and discuss. Before I touch on video games specifically, I hope you’ll indulge a brief aside about vocabulary as it relates to Nineteen Eighty-Four.

Newspeak and thought control

In the novel, protagonist Winston Smith lives in a totalitarian state (known as Oceania) ruled by a single political entity simply known as “The Party”. Much like the totalitarian states that we are familiar with today (Russia, for example), The Party utilises sophisticated propaganda as one of many methods to control and influence the populous. But in Nineteen Eighty-Four, the dissemination of false news dovetails with a broader — and perhaps more sinister — effort to physically control language itself through the application of Newspeak.

Newspeak is the official language of the state, and is subject to continuous revision by The Party in an effort to progressively restrict both grammar and vocabulary. The aim is to ensure that all language — and therefore, all possible thought — is guided through a strictly-controlled funnel that aligns with the principles of Ingsoc (the ideological orthodoxy of The Party). Consequently, unorthodox language and thought is avoided to the maximum degree possible.

A great example can be found on the relevant Wikipedia page:

The Oldspeak word ‘free’ existed in Newspeak, but could only be used in terms of something not present, as in the sentences “The dog is free from lice.” and “This field is free from weeds.” Politically, the word free could not denote free will, because such a humanist concept does not exist in the society of Oceania. The linguistic design of Newspeak is for thought control, by diminishment of the user’s range of thought, which is realised with a minimal vocabulary of limited denotation and connotation; hence words such as: crimethink (thought crime), doublethink (accepting contradictory beliefs), and Ingsoc (English Socialism).

Take a moment to consider the implications of this. The idea is very simple: if you have no way to articulate a particular concept through language, then your ability to think of or imagine the concept is either diminished or entirely negated. There is a very real sense in which your articulation of the thing enables your conceptualisation of the thing.

In this way, I see language and its constituent parts (vocabulary and grammar) as tools that can enable us conceptualise something with greater or lesser degrees of fidelity.

Pulling the strings

I might be drawing a long bow, but I think there’s a clear comparison to be made with video games here. If the game is the thing or the concept — and if we are interacting with or interpreting that concept — then surely the controller is our vocabulary and grammar. You might argue that there are other elements at play (like graphics and sound), and you’d be right. But your ability to manipulate the game (to not simply view it passively, but to “conceptualise” it in literal terms) really comes down to the tools you’re using to physically pull the strings.

If you accept this argument, then my underlying thesis might be that much more salient: game controllers can fundamentally limit or expand the kinds of experiences we can have with video games. To some degree, I’d argue that controllers influence this even more than raw processing power.

Here’s a little thought experiment I went through, which might help to underscore the point. It relates to the first time I played Super Mario 64 (this works especially well for me because I’m ancient enough to remember the pre-N64 consoles when they were still new).

I will never forget the moment when Mario triumphantly leapt out of the warp pipe at the beginning of the game, large as life, barely a moustache-whisker away from the camera lense.

Mario in 3D! Wow!

And then I tilted the little analogue stick on the Nintendo 64 controller (which, at first glance, seemed like a completely bonkers way of doing much of anything in a video game). It all felt so new; I remember tentatively nudging the stick upwards.

Mario is…tip-toeing?

Now a more confident tilt.

He’s walking!

And now — complete with an ear-to-ear grin — full tilt.

He’s running! Look at his little arms and legs go!

It seems so simple now because it’s totally and utterly granted in virtually every modern game. Back then, though, this felt like magic!

I’d seen 3D games before, having played a Sega Saturn well before I ever laid eyes on a Nintendo 64 in the flesh. Although Super Mario 64 was visually more advanced than anything I’d witnessed on the Saturn to date, the concept of 3D graphics wasn’t new to me. Mario felt so new — so different, and so liberating — because until that point, I’d never manipulated a character in a 3D space with true 360 degrees of movement and the precision that analogue controls afford.

As much as the Nintendo 64’s processing power enabled those large and beautiful worlds to unfold before my eyes, it wouldn’t have meant anything — and it wouldn’t have been so utterly delightful — without that revolutionary controller.

Gaming history buffs may be tempted to note that Sega actually beat Nintendo to the market with its 3D control pad for the Saturn. That’s true, although I’d say it was the combination of Super Mario 64 and the Nintendo 64 controller that represented an enormous leap forward for video games.

Survival of the fittest

Throughout video game history, there have been many unique controller designs, but very few become mainstream or maintain their influence across many generations of games.

The Atari 2600 Paddle Controller is one of many designs that was popular for a brief window and was then superseded by other designs. This particular controller debuted alongside Atari’s Pong (arcade version), and a similar design was later used across numerous platforms (including everything from the Magnavox Odyssey to the Apple II. This form factor actually survived for some time and was well-suited to certain genres (including various driving and sports titles). The underlying technology underwent several revisions, though, because the original potentiometer-based design loses accuracy when the potentiometer contacts become worn or dirty. Although this design appeared in various forms throughout the years — particularly in arcades — it never became a mainstream design for most games (in contrast to something like the D-Pad, which is still an important standard, even in an age of 3D analogue-based game controllers).

That said, not all game controllers were necessarily intended to unseat the mainstream designs. The highly-unique Dreamcast Fishing Controller, for example, was really only ever designed for use with Sega Bass Fishing, although it was compatible with some other games (such as Virtua Tennis).

Motion is the future…right?

Not so long ago, you might have been tempted to claim that some form of motion control would unseat the humble — but ubiquitous — analogue stick. That time may have long since passed, but when the Wii first exploded, the survival of “traditional” controller didn’t seem so certain. Not only did the Wii become a massive global phenomenon, but it did so in large part thanks to the innovative Wii Remote controller.

When I first played WiiSports, I definitely felt that the experience was new; arguably as new as the feeling I’d first had with Super Mario 64. What was different, though, was that the Wii Remote wasn’t some complex controller that only long-time game enthusiasts could enjoy; it completely levelled the playing field in a way that I don’t think I’d seen since the original D-Pad design launched. So-called “lapsed gamers” (people who hadn’t played games in years, perhaps since the 16-bit era) and “non-gamers” (people who had never played a game at all) apparently made up a large proportion of the Wii’s audience. This made sense: the Wii Remote relied on natural, physical movement rather than complex button combinations.

It didn’t take very long for Nintendo’s competitors to not only recognise this “blue ocean strategy”, but to begin responding in kind.

Sony more or less mirrored the Wii Remote with the PlayStation Move, which is functionally quite similar to the Wii Remote, except that it is underpinned by different technology (it tracks its position in 3D space by having the PlayStation Camera track the glowing balls atop each controller).

Microsoft went in a completely different — and to their credit, quite bold — direction with Kinect. Originally unveiled as “Project Natal”, Kinect promised a very different future for video games: digital characters would be able to converse directly with the player and interact with them naturally without the need for any kind of physical controller, thanks to a highly sophisticated motion tracking/image recognition software.

I think the best representation of what Kinect promised was the E3 2009 Milo demo video. In recent interviews, Peter Molyneux was asked about Milo and he said that as development of Kinect progressed, the technology was progressively stripped back in order to achieve a reasonable price point at retail. The rest is history, really: Microsoft valiantly continued on with Kinect for a while (even including it in the first-generation Xbox One hardware), but it was ultimately relegated to the status of a pricey (and optional) accessory.

Motion control has never really seen the ubiquity it achieved during the Wii years. Even the Wii U — which did support the Wii Remote — ended up focusing on a controller/tablet hybrid design, which didn’t set the world on fire (although the concept of using touch screens for games has become as ubiquitous as the D-Pad, if you consider the proliferation of games on smart devices like phones and tablets).

New frontiers

It was only quite recently when entirely new video game experiences could be achieved solely by improving graphics technology. If you move from NES to SNES, for example, games don’t just become prettier; the palette for developers becomes sufficiently larger to offer gameplay designs that simply weren’t possible before. Whether it’s about adding more characters to the screen, or creating a realistic feeling of driving through the use of technology like Mode 7, increasing hardware power allowed for new experiences to emerge.

I would argue that we are no longer living in a world where iterative advances in graphics technology alone can foster completely new gameplay experiences.

At this point, it’s worth going back to the Sega Saturn vs Nintendo 64 example I mentioned earlier.

Compared with its predecessors, the Saturn offered a new paradigm in terms of graphics technology — for the first time in a home game console, players were able to truly explore 3D worlds in a fidelity that hadn’t been previously available outside the arcades. I remember playing Nights into Dreams and being completely blown away at how stunningly beautiful it was at the time.

Why is it, then, that playing Super Mario 64 felt like crossing the Rubicon into a completely new world?

It was not just about visual fidelity — it was about the intersection of advanced graphics and innovative 3D controls that dovetailed to create an experience which looked and felt like nothing else.

This is where game controllers — or, more generally, the way we interact with games — play an increasingly important role.

Consider that games like Half-Life: Alyx are best experienced — and arguably only truly possible — thanks to highly-sophisticated VR controllers that provide a degree of movement and dexterity in 3D space which has never been possible before. Of course, the graphics technology and the VR headset are also crucial. But imagine playing a game like that with, say, a traditional Xbox or PlayStation game pad. Your relationship to the in-game action would be abstracted by many orders of magnitude.

Even “traditional” controllers continue to evolve in an effort to establish finer and more nuanced points of connection between our hands and the digital worlds we’re interacting with. Sony’s DualSense controller for PlayStation 5 is a great example. Not only does the entire pad apparently feature a form of force-feedback that is even more advanced than we see on the Nintendo Switch Joy-Cons, but the analogue triggers are able to “push back” against your fingers (one potential use case here involves, say, drawing back on a bow and arrow, where the trigger will provide greater resistance a the tension in the string increases).

It’s easy to forget just how much controller design influences the way games feel and the possible range of interactions we can have with them. If you’re in any way tempted to underrate the importance of this, I recommend going back and playing Super Mario 64 DS using only the D-Pad. The central importance of the analogue stick immediately becomes apparent. It feels something like wearing heavy gardening gloves while trying to do fine calligraphy; clearly, it’s the wrong tool for the job.

I think it is also important to understand that when we talk about user interfaces in games, we aren’t just talking about the GUI (graphical user interface) within the game itself. All components that connect player to game are part of the interface — including both the controller in your hands and the technologies through which audiovisual information is being relayed to you (your screen and speakers, in other words).

In the same way that Newspeak applied deliberate constraints to vocabulary as a way of restricting possible thought, the mechanical constraints of game controllers ultimately determine the range of possible actions within a video game. It may be tempting to think that controller design has reached its zenith with dual analogue sticks, four face buttons, and comfortable shoulder triggers. But I’d argue that the industry should continue to explore both evolutions of this model as well as entirely new models. So, in conclusion, I feel the need to borrow a concept from Satoru Iwata here: we live on a planet we call “game controllers” — but what other planets could we discover? It’d sure be fun to find out.