User Interface Mapping - the need for standards

23rd December 2002

Current UI Mapping Systems

Most of the games I have looked at recently implement their own mechanism for mapping user interface actions to in-game actions. Some developers do a good job, but the majority do not.

My definition of a good job is one where the developer allows maximum freedom for the user to map UI actions (keypresses, mouse movement, mouse button presses, joystick buttons etc) to in-game actions (jumping, running, shooting etc). The developer must assume that an advanced user may want to link these two sets in complex ways, going beyond the mappings that the developer is familiar with.

There is a clear need for this in most games. Effectively the UI mapping ability must grow along with the users level of expertise in the game. As users become more sophisticated, the demands that they wish to put on the UI system will become more complex - fairly soon they will be going beyond anything that was tried (or imagined) by the developer.

Unfortunately, most games provide little or no ability to extend the UI mapping beyond the fairly basic schema imposed by the developer, and as a result the gaming experience is reduced for advanced players - exactly the same group that form the fan base which can turn a game from a short-lived novelty into a long-lived obsession.

It is clear to me from first-hand experience that many developers impose their views about the UI mapping on the player because (perhaps subconsciously) they believe that they know best what mapping works for their game - after all, they developed it, and have spent some time "playtesting" it. After a game is released, however, history repeatedly shows that it is the gaming community that takes the game from its initial state and pushes new and innovative ideas in the search for an even better game. The gaming community is focussed on "making the game better", whereas the developer will be focussed on "making the game better, within budget, appealing to a mass market and selling enough copies to finance the next game". These are two different goals.

Developers should recognise that it is the gaming community - and fan base - which will generate the best ideas about how the user interface for their game should look, and it is the developers responsibility to provide a UI mapping facility that is powerful enough to allow the players to experiment with maximum freedom.

The bindings provided by default should be seen as the developers "first guess" - something to be used by new or uncritical players until they start to experiment on their own.

The UI Mapping Problem

UA: User Actions - actions performed by the User via keyboard/mouse/joystick etc.

GA: Game Actions - In-game actions performed by the players persona or avatar as a result of some UA. e.g. Walking, jumping, shooting etc.

UI Mapping is the task of converting UA into GA, and converting game feedback into user feedback (e.g. via force-feedback).

User Actions fall across a wide spectrum - from the simple press-a-key-to-attack where there is a direct 1:1 mapping from UA to GA - to more complex combinations where multiple UA map onto a single GA, or a single UA maps onto a collection of GA to be executed either sequentially or simultaneously, as appropriate to the game in question.

*Note that the ability to alter the UA:GA mapping is not intended to provide the player with "superhuman" powers, but rather a means to change the impedance match between the game and the players perception of the game. It also allows the player to set a familiar collection of mappings for all their games, again reducing the impedance mismatch between their perception and the game "reality" because all their games in a particular genre can use a common set of interface actions.

The set of "User Actions" would include all basic actions that the user can perform - keypresses (single and double), mouse/joystick/wheel/pedals axis and buttons, etc. Pressing a key might be indicated differently to releasing a key - consider this hastily constructed UA syntax example:

Syntax		User Action

+S 		Press S
-S		Release S
S		Press and Release S as a single action
DOUBLE-S	Double-tap S
+SHIFT-S	Press S while holding the SHIFT key
+S-MOUSELOOK	Moving the mouse while holding S
+CTRL--S	Holding the CTRL key while releasing S
+TAB--		Press and release "-" while holding the TAB key

The point here is that the UA syntax expresses ONLY combinations of physical actions that the user can perform.

Here is a sample collection of "Game Actions":

Syntax		Game Action

attack_1	First (primary) attack
attack_2	Secondary attack
attack_3	Third attack... etc
openmenu_1	open menu 1
closemenu_1	close menu 1
closeactivemenu	close the currently active menu

FPS games might need:

move_forward
move backward
strafe_left
strafe_right
turn_left
turn_right
jump

RPG style games:

quaff_1		Quaff potion number 1
quaff_health	Quaff a health potion
cast_slot_F1	Cast spell in slot F1
inspect(item)	Inspect "item".
walkto(location) Walk to the nominated location.

A scripting language will be necessary to glue the UA's and GA's together. My language of choice here would be Perl, but I guess other languages might also do the trick. (I'd also have various properties bound as hashes - like the contents of the players backpack(s), their current equipment etc.)

UI Mapping Example

For a quick example, I will use the RPG / D&D genre (e.g. Dungeon Siege, Neverwinter Nights, etc). A common activity in this genre is "collecting loot" - picking up items off the ground, inspecting them and adding them to your inventory or discarding them if they are not useful.

In pretty much all game systems, you are forced to pick the item up before you can inspect it - the game will only allow you to see the properties of something that you are holding or carrying in your "inventory". The concept of inspecting something before you pick it up is mostly non-existent, and yet this should be a common activity. Consider how much easier it is to "inspect and ignore" an item as compared to "pick it up, find somewhere to put it, inspect it, drop it again" sequence forced on us by most games.

If your inventory is full, you also have to add the extra steps involved in "dropping something you are carrying to make room for the new item" and then "picking up that item again".

Something that ought to be a simple activity turns into a hurdle.

Now, contrast this with what the player would imagine him/her self doing in the "real" world. Surely you would walk over to the item and then inspect it carefully _before_ deciding whether or not to pick it up. Players who compulsively pickup items are usually in for a nasty shock in the D&D genre.

(Of course, a magic items' real nature may not be revealed until the player actually picks it up, but that's ok - it follows the players understanding of expected behaviour).

Allowing the user to alter this behaviour requires that some GA directives exist for addressing one or more "ground" locations around the players current location in the same way that inventory slots are addressed - indeed the ground locations adjacent to the player should be seen as simply another part of the players current inventory, with a possible time penalty incurred when they are used.

There ought to be a way of expressing the "walk over to this item and examine it" action that is distinct from "walk over and pick up this item".

A mapping might look like this:


UA				GA pseudocode

SHIFT-MOUSEBUTTON1	if (selected_item and onground(selected_item)) {
				walkto(selected_item);
				openmenu(INSPECT, selected_item, PICKUP | SELECTION_CHANGE);
				}

MOUSEBUTTON1		if (selected_item and onground(selected_item)) {
				walkto(selected_item);
			   	pickup(selected_item);
				}
This does the trick nicely. A shift+mouseclick on a ground item will cause the player to walk over and inspect that item, and a simple mouseclick will pick it up.

We have also set the close-on properties for this menu to indicate how & when it should automatically close. In this case, either when the player picks up the item or when it loses focus (the player moves their focus by pointing & clicking somewhere else).

It is easy to imagine extending this code - for example if the item is a health potion, and you are in the midst of combat and low on health, then the script might pickup and quaff the potion automatically. After all, this is exactly what you would do in that situation, right? The point is that you (the user) should have the ability to decide how the game interface ought to work, overriding the defaults set by the developer.

Standardising the UI Mapping System

There is another reason why most games do not provide a full-featured UI mapping system. Developing one from scratch, including creating a custom scripting language to hold it all together, costs a significant amount of money.

The cost is high enough to make a complete system too hard to justify in the context of the overall game development budget.

The problem with this mode of thinking is that each developer is considering creating these systems from scratch. This is akin to the situation in the video card arena of the early 1990's, where every manufacturer of video cards had their own custom API, and game developers had to explicitly code and support each video card manufacturer separately. Eventually there was a change in the approach of video card manufacturers - they added an extra layer between their hardware and the game developer to allow the standardisation of the video API. Now there are just 2 major video API's in use: Direct3D and OpenGL.

This shift in video standards has allowed game developers to put more of their effort into game content without having to worry too much about the low-level details of the graphics system. (Well, this isn't strictly true - there are still plenty of incompatabilities between cards conforming to these "standard" API's, but at least the basic stuff seems to work as expected.)

A similar shift must happen in the User Interface arena to allow games to move up to a higher level of usability and to free developers to put more of their energy into creating game content.

Of the three major pieces to the UI Mapping system ( UA definitions, GA definitions and scripting language) the UA definitions and scripting language can be common across games, leaving only the GA definitions and actual scripts to be written for a particular game.

This makes a lot of sense - after all, the UA definitions relate to physical items such as keyboard and mouse - there is nothing specific to any game in them. The scripting language must also be game-neutral - my personal choice for the scripting language would be Perl. I would advocate that you will not find a more powerful, flexible and cross-platform language more suited to the task.

( Perl is my personal choice - no doubt there will be people advocating Python and other languages (scheme?). It doesn't actually matter all that much which one is used, provided that it is standardised across a lot of games, and that it is truly a general-purpose, powerful language. I happen to have a lot of practical experience in using Perl for this sort of thing, and I can imagine how much nicer the gaming world would be if I could write perlscripts to customise my interactions.. )

The only part of the picture that the game developer should have to focus on is the GA definitions, and writing one or more Plugins to load these definitions and their associated callbacks/actions into the interface system.

Aside - NeverWinter Nights

Developers should not have to expend their energy creating half-baked, proprietary scripting languages that constrict the end-users ability to enhance the gaming experience. One game - NeverWinter Nights from BioWare - comes strongly to mind here. For all the features in the scripting language that they provide, the language has a glaring lack of support for array data types, no ability for a module-developer to specify their own editor of choice inside the IDE, and no way to work on the scripts outside their integrated environment without risking disaster. (The IDE automatically packs up and deletes your scripts when you close it).

When I first experimented with the scripting language used in NWN, I thought it might get around some of its faults by writing some high-level routines in Perl and then creating a back-end translator that would convert my Perl into NWScript. However, due to the issues above, and the rampant number of bugs in the NWScript language and functions that existed when the game was first released, this turned out to be infeasible. This is an example of a developer forcing a straitjacket view of "Interface" onto the community. You either create NWScript the way that BioWare intended, or you don't create it at all.

When I look at the amount of effort that BioWare has expended to create this proprietary, substandard language it makes me feel quite frustrated. The world does not need Yet Another Scripting Language.

When I look at the energy put into the NWN mod community to develop custom add-on module for NWN, I can only wonder at how much more might have been attainable if BioWare had chosen to use Perl as their language, and created a collection of NWN Perl Modules to expose the underlying game API and systems. The savings in developer time alone must surely have added toward improvements in other areas.

Proposal - a Standard UI Scripting Interface for Games

There should be a standard library that developers can incorporate into their games which provides as much as possible of the "common" components of a scriptable, powerful UI language. Potential scripting languages already exist - I would advocate Perl, other people may advocate other languages - and a comprehensive set of User Action definitions is almost trivial to create.

A developer would only have to develop a collection of plugins for the language to provide the proprietary low-level Game Actions and their hooks into the actual game API.

Thoughts about the Future

I can imagine a future where the User Interface and scripting system is separate from any particular game, to the point where is will be possible to combine gaming universes in ways that are flat-out impossible at present.

How would you like to play NWN or Dungon Siege, but using the same first-person interface as CounterStrike?

Or load the pack-mule from Dungeon Siege into NWN to help carry your stuff...

Maybe even have players running DS, NWN and other games all adventuring together in the same online universe?

The best way to move forward as I see it is to break open these closed, proprietary systems and add a new layer that provides for real power, flexibility and the ability to grow.

Anthony Wesley, 23rd December 2002