Figuring Out Enemy Triggers

To what extent should an intelligent player be able to work out the triggers and AI routines that govern enemy behaviors?

This is a question I’ve been pondering some time. This entry has been going back and forth in my head for the several years now (!) so I’ve decided to finally upload it as something more akin to a collection of ideas and questions than a particular conclusion. In URR we’re now implementing conversation and movement AI, before too long we’ll be adding combat AI, and I’ve been wondering how transparent a game’s AI should be. Should the player be able to figure out exactly how it works and be able to predict what it’ll do, or be able to work out how it works but be unable to predict (due to schotastic elements), or should I base it on some other rationale? To think about these issues I’m going to start off with a few example of AI routines being deduced, the benefits and weaknesses of these deductions, and then think about the best way to make an AI interesting and unpredictable: does this come down to random decision-making, to such complex decision-making the player can’t figure it out, or decision-making that can be understood but is too difficult to act upon (the game is too fast, or the variables too many?). As I say, this entry is a large collections of thoughts on this topic, but I don’t think I’ve yet reached any conclusions, so any thoughts you folks all have will be greatly appreciated in the comments section down below.

Metroid Prime

In the original Metroid Prime one of the latest (and by far the hardest) boss is known as Omega Pirate, shown in the picture below. In the fight he teleports around the arena and recharges himself in these three pools around the arena. When you do enough damage to him during a recharge, he will teleport to another pool, and so on until you fail to do a certain amount of damage and he will then stop teleporting and resume the normal fight.

ACmap

At first many players thought it was simply random – it was easy to spot that he never appeared in the same pool twice, but the future pool seemed random. However, someone eventually – after years, I believe – figured out how the game decided, and it actually isn’t random it all. Omega Pirate never appeared in the same pool twice, and never in the pool you were nearest to of the remaining two. This meant that when he vanished from Pool A, standing near Pool C would guarantee he’d spawn at Pool B, and vice versa for B/C. This makes the boss fight significantly easier (especially on speedruns, from what I’ve seen) and even if playing “casually” it’s a massive help – not just can you shoot earlier because you don’t have to look around to figure out where Omega Pirate is going to respawn, it helps you control the boss fight more generally; you can decide when and where to kill the small enemies that spawn throughout the boss fight and position yourself well each time he reappears. In this case, figuring out the AI makes the boss fight undoubtedly easier, and although the actual algorithm is very simple, there were enough possible variables that it took some time until anyone cracked it.

Dark Souls

The Souls games are known for having a large number of bosses with (generally) unique attacks and patterns. Although no Dark Souls boss is fully deterministic – or if they are, the determinism comes down to pixels, or frames, or similar, and likely could never be achieved by a human player – many can be exploited. There are three “Demon” bosses (Asylum Demon, Stray Demon, and Firesage Demon) who share a lot of attacks, including one where the boss flies upwards and does a “butt slam”.

Buttslam

This attack has a very clear tell, is very slow, very easy to dodge, and gives you a good second or two to attack them once they’ve landed. In other words, it is an ideal attack for the boss to do. By contrast, both the Stray Demon and the Firesage Demon have huge area-of-effect attacks, and are fought in small arenas. You don’t want them doing these. Although the system is not perfect, running directly against the belly of the demon when they are resting on the ground has a very high chance of encouraging them to do the butt-slam, and I have often got them into AI loops of 10+ slams in a row, all but trivializing the boss fight. Other bosses have equivalents, such as attacks they might do if you are under/over a certain range, or performing a certain action yourself. One or two bosses are very clearly specifically coded to come out of their “passive” animation and attack the player if you pause to heal (e.g. Artorias), therefore likely mitigating the entire effect of that heal (or doing so much damage you are left worse off than before). In other cases running directly towards them appears to trigger certain actions (e.g. Seath, Smough or the Iron Golem), or being in a certain position relative to the direction the boss is facing will affect attacks (e.g. Kalameet), or knocking the boss below a certain HP (e.g. Manus), whilst others seem to have hidden timers running under the hood for certain attacks before they are allowed to use them again (e.g. the Bed of Chaos’ firestorm). In this case the AI algorithm is determined by your movement and actions related to the boss, not related to the map. The few enemies that do have some pattern other than randomly selecting attacks are generally based on where you are in relation to the boss, or what action you’re doing. Some of these are only “negative” – such as Artorias always attacking when you heal, as long as he isn’t already doing an attack – whilst others, like the Demon bosses, can be useful.

Angband

Angband, one of the classic roguelikes, has a form of AI that has become notorious among roguelike-players because, although broadly deterministic, it is exceedingly hard to outsmart. Rather than randomly selecting from a spell list or a selection of special moves, in most Angband versions enemies aim to choose the most intelligent and appropriate attack from their repertoire for the situation. Whereas in Dungeon Crawl Stone Soup spell-casting enemies, for example, are sometimes balanced by giving them a “spell set” which might have some stronger and weaker attacks (for example, Gloorx Vloq’s Invisibility and Poison Arrow are all but meaningless by the time you encounter him), Angband’s AI always seeks to choose the most painful option for the player. Even if AI decision-making can be perfectly understood, therefore, that doesn’t mean it needs to become trivial. If the decision-making is so strong that the choice the AI makes is always challenging, whilst you may be able to “predict” it (or at least make an educated guess), that doesn’t mean you’ll always be able to deal with it.

ACmap

So it seems fully transparent decision-making need not be simple. If you take the DCSS route and we know the enemy will pick from one attack of four, if all four attacks are very different and demanding to avoid/mitigate, that might be more difficult than an AI that specifically chooses its attacks for the situation. Alternatively, if we follow Angband and we could try to make an educated guess about what attack the AI is going to choose – either because of our actions, or position on the map, or the tactical situation – it still might not be trivial to deal with either due to your character setup or the input skill required to survive. Although the DCSS AI can sometimes work to provide a challenge due to its random behaviour, the Angband AI is more interesting and significantly closer to the question of how predictable it should be, and whether it matters if you can predict what the AI’s going to do. In this way, I’d suggest the Angband AI tries to behave “like a human” – it judges the situation and chooses the best selection. But are players necessarily harder to judge than an AI?

Counter-Strike?

Take a Counterstrike:Source example I played several years ago, which for some reason has always stuck in my head. In the picture below I was the green circle, a little inside the garage on cs_office; my enemy was the red circle, waiting out in the snow. My enemy had a superior weapon and – since I have barely played in the last five years – undoubtedly superior aim. We exchanged momentary fire before I ducked back in. Up the staircase on the left of this image is a window looking out over my foe. I backed off originally intending to move up to that window, but then it struck me that would be too obvious. Hoping my foe would look up there (as in the red dotted line), I instead moved back out. He had indeed started looking up at the window and I was able to pick him off. Whilst not the world’s most brilliant strategy, I was certainly satisfied at having tricked my foe, and whilst carrying it out I felt I had a decent chance of the ruse succeeding. Although I had never played with this player before it seemed like a logical tactic to attempt.

Office

But here’s the interesting thing. Imagine if we codify this into an AI routine. If the AI sees you moving out of its line of sight, maybe it takes note of the direction you’re moving and sorts through the areas you could potentially move into. When it finds the first area from which you could see it and it could see you, the search routine stops and it aims its gun at that point for x seconds until either you appear, or some other trigger overrides it and draws its attention. On one level, it seems like this could make for quite a clever AI. It looks at where you might move to and sets itself up accordingly when you try to flank it, preventing you from just doing a quick loop if some scenery before killing them from behind. However, I think it would be quite easy to spot if an AI always acted this way, and certainly easy to take advantage of in exactly the same way I did. Such an AI, although it might have a fairly complex piece of code behind its where-might-you-next-appear identification routine, would be trivial to understand, and trivial to defeat. Player information would be absolute – moving out of sight causes your enemy to look for other places you might emerge – and provides far too much information to the player on the AI’s systems.

In the situation I fought, the difference was is that I couldn’t be confident my human adversary would do this. My enemy might have thought “He thinks I think he’ll be at the window, so I’ll actually keep my guns trained where he was a moment ago”, and would have been waiting for me. As it happens, they didn’t. In poker terms this is “levels” of thinking – what’s my hand, what’s their hand, what do they think my hand, what do they think I think their hand is… and so on. Perhaps a stronger version of such an AI would flip a virtual coin when you go into cover; half the time they search for where you might appear, and half the time they’ve keep their eyes on where you just vanished. This comes with its own issue – most players, I wager, wouldn’t bother trying to flank, and therefore if they just re-emerge from the same cover, half the time they can guarantee their foe will be looking away from them. But if we take account of this and decide that the AI will only look for your appearance elsewhere on the map 10% of the time so that players who just rush back out will normally get shot, then any savvy player will adapt. How can we keep an AI challenging in this kind of scenario? I think the key here is that landing the shot – especially in a game at the speed of CS and where players have as little health – is not trivial. Even if you’re using a wallhack and you know exactly when and where someone is going to turn around the corner, there’s still no guarantee you’re going to land the kill against a skilled player. So perhaps how demanding your response to the AI choice affects how much it matters if you can predict that choice?

The Solution?

I think we can consider this to be a flowchart. Firstly we can ask – is this game turn-based, so you have a potentially infinite length of time to think about your solution, or is this real-time and you need a reasonable level of twitch/reflex skill in order to survive? If turn-based, an exploitable AI will result in a very mundane experience; put in the correct inputs (which is trivial, since it is turn-based) and you get the output you know you wanted. You’re then left with two solutions. Either you take Angband’s method and make the AI always choose what it considers to be the best move – not a move you have “induced” in the AI like in Dark Souls – and make sure that move will always (or will a lot of the time) be challenging to handle. Or you take the Crawl route and have full randomization of attacks; you know the possibilities of their attacks but you don’t know any given specific incoming attack so you have to respond. For the DCSS method it doesn’t matter that you know how the AI functions (total randomness), and for Angband knowing how the AI functions is actively an important part of the game – even when you know how it functions the game remains highly challenging due to how cleverly the AI behaves. Alternatively, if the game is real-time, you can make your AI’s choices a little more predictable so long as dealing with them isn’t trivial, or figuring out what determines their choices isn’t too simple a process.

In many ways Dark Souls’ AI takes the best of both worlds – attacks are predominantly “random” (from the player’s perspective), and are generally challenging to dodge perfectly. You can exploit them to an extent as an effective strategy, but you still have to execute your avoidance of the attack correctly. For a roguelike, I think Angband’s method works brilliantly so long as the AI really can make those kinds of decisions well, and the decisions aren’t sufficiently trivial that you can immediately negate the severity of the AI’s choices with your own decisions. If the AI’s most devastating attack is hardly different from it just flipping a coin, there’s no threat; but if the AI will always look carefully at its situation and pick the toughest possible choice, there is.

Ultimately, I don’t think I’ve really found the answer in this post – but hopefully I’ve related some of my thoughts about the complex intertwining of predictability, exploit-ability, and the ease or difficulty of playing against certain kinds of AI actors in certain contexts (and how the real-time or turn -based nature of the game is undoubtedly important). Different games handle this in different ways, and for the main part of URR’s AI (the conversations) and the future part of URR’s AI (the combat), I haven’t yet decided on transparency and how it will impact gameplay. As I’ve mentioned in recent entries, conversations are definitely not going to have any kind of counter or visible metric for representing mood – it’s up to the player to figure that out – but that’s only one part of the overall AI transparency questions. It remains to be seen how much of the rest of the AI’s decision-making we want to show, and how much should be kept hidden, and how these decisions are going to wind up shaping the player’s experience of URR.

Be Sociable, Share!

7 thoughts on “Figuring Out Enemy Triggers

  1. Sirlins articles are a required reading for this:
    http://www.sirlin.net/articles/solvability

    I think there are a couple of axis you can put this on.

    In close combat with a more limited set of moves its best to used weighted randomness and maybe some attack patterns.

    In more tactical situations at range with multiple opponents with abilities and a wide range of magic abilties, skills, items and party members its better to be more direct and predictable.

    You also take into account what the intended difficulty is and what are the stakes.

    In a game with permadeath and deadly combat like impactful wounds and high risk it might be best to make things more deterministic and predictable with simple patterns.

    Information gathering also could play a role, if you can give players some hints about the opponent before combat is started it can change things up rather then always playing a defensive strategy.

    Restrictions are also interesting where you can deny the enemy some options in some conditions.

  2. Interesting article. Comes to a kind of basic conclusion: it’s nice to have bosses/enemies that are semi-random, with a degree of predictability but not complete predictability. Also that it’s nice to have attacks that players can exploit as long as they also need to avoid it. But you cite nice examples.

    I wrote an article sort of similar to this a while back, and I have a different premise that I think might add to your work here. The idea is that every behavior of the AI exists to simultaneously limit player actions, to threaten them in some way, make them deal with what the AI is throwing at them, but also gives the players a way to exploit the AI back, and never to let either of these be too powerful.
    https://critpoints.wordpress.com/2015/05/18/taking-apart-stealth/

    I think trouble comes from viewing enemies in the chess AI sense, where it’s trying to play well. If you did this for a fighting game, then you might get Street Fighter Alpha 2 Akuma, who walks up to you and throws you if you block, does an invincible move if you attack or jump, and blocks if you do an invincible move. Since he can react on the frame and the player can’t, he has an unbeatable pattern here. It’s more helpful to view AI as something that encourages to weigh their options and use them proportionally to each other. That requires them to change tactics in accordance to what the AI does, but also try to lure the AI into doing certain things sometimes. Take inspiration from the habits humans play with so that you can give the player more ways to manipulate the AI, and the AI more ways to attack the player. But also make sure most everything is communicated clearly.

    Of course, old games didn’t use this type of AI at all. They did perfectly fine with simple deterministic AI used in combination with level design, look at castlevania 3 or ninja gaiden. Layering threats and putting them on asynchronous cycles can also make interesting challenges for the player, whether they react to the player’s actions/position or not.

    I don’t think Sirlin’s articles are totally related, as those function mostly based on multiplayer games, not singleplayer ones. If there’s a rock paper scissors element that bots need to make use of in a simulated multiplayer game, I recommend using a weighted RNG, maybe one that shifts over time, so players can feel like they’re noticing patterns and taking advantage of the AI. For single player games, enemy actions should never be random, should always be reactable in some way. In multiplayer you can’t make the same affordance unfortunately. Players can predict players fairly well with practice, but predicting RNG is impossible.

    • Thanks; yeah, ultimately I knew this piece didn’t have a fixed conclusion, as I think the issue is way too complex to explore in a single blog entry, and I honestly see the value of both sides of the argument. Thanks for the link: definitely an interesting piece, and I like the idea of the balance of preventing either from getting too far ahead of the other, as it were. And as you say, many genres and styles of game actually do perfectly well with extremely simple AI because nothing more complicated is actually needed. In many ways I think more study of how tremendously simple AI can generate interesting challenges would be a really fruitful direction to explore…

      • If you want simple AI to study, I’d recommend Castlevania 3, Ninja Gaiden, AM2R, Contra, Metal Slug, and whatever the best shmup games are on that one shmup forum you always run into in searches.

  3. I’d like to add, bringing some “learnability” into the equation can make a big difference – in the CS example above if you regularly pop back out the door rather than go to the next potential pop-out point then maybe that would become “known” about you, and weight the response appropriately? Could this form some part of the calculation about best response on the part of the AI?
    It’s certainly a consideration on the part of playing against other humans – both when trying to determine what their future actions might be, and also on your own actions – how many times have I moved on, shall I move on or stay – and I’m sure that keeps the challenge more alive.

    • Yes, definitely, and if I ever write a second part of this I think that’s what I need to focus more fully on: the ability for the system to respond to the player, though this is obviously extremely rare in AI as it currently stands…

Leave a Reply

Your email address will not be published. Required fields are marked *