I've been thinking about ways to improve the team mate stats, and I've come up with a few ideas. You've also given me a few.
if a pokemon is not on the teammate list, doesn't that mean that it is a rare partner? i would think that you could assume the multiplier is some arbitrary number like .9 to reflect that.
What I was considering is looking at all the usages that the stats do give me (top 20 used with it, top 10 by change up, top 10 by change down, so 20 to 40 Pokemon), and then just kind of consider every other Pokemon as being equally likely to appear. In other words, if 70% of all team mates can be accounted for by the listed Pokemon, then weight the remaining Pokemon equally down as being just 30% likely. I have to consider carefully how to weight that exactly, though.
something else came to mind when i read this. usually if i'm battling and i see a pokemon like forretress, i go ahead and assume that my opponent doesn't have a similar pokemon like tentacruel. how can you make the ai pick up on that?
I had that problem for a little bit, because Tentacruel doesn't appear on the usage stats of Forretress. However, Forretress appears on the usage stats of Tentacruel. After a lot of trying to fix a bug that didn't exist while trying to implement this, I realized that the 'bug' that the multiplier for Forretress being on the team with Tentacruel coming out to the same as the multiplier for Tentacruel being on the same team as Forretress was actually not a bug. If Tentacruel make Forretress about 40% as likely, Forretress makes Tentacruel about 40% as likely, too. It took me a while to realize this, but it turns out that because I'm using a multiplier method here, that works out both directions. This allowed me to take the team mate stats that normally only give somewhere between 20 and 40 data points for 100 Pokemon (for a total of at most 4000 data points) and end up with 5168 data points (unique data points, so it doesn't have Abomasnow paired with Walrein on there twice like the stats on the site do). If I wanted, I could 'compress' this to half that by not storing Walrein with Abomasnow, but it's easier to parse this way. Now if it sees either Tentacruel or Forretress, the other one is 40% as likely as it was before it saw that Pokemon.
couldn't you simply subtract the lead usage counts from the total usage counts? instead of referring to a list of pokemon by usage, it would be a list of "usage as non-leads". obviously stuff like azelf and aerodactyl will be far lower on this list.
Unfortunately, that does not work. The reason for this is that team mate stats don't discriminate between lead team mates and non-lead team mates. Fortunately, the real solution is almost as easy. I simply apply a non-lead multiplier after seeing the first Pokemon to my table of probabilities. In other words, if Azelf appears as the lead 40% of the time, I just multiply the probability of Azelf appearing by .6 (which, because everything I do is multiplication, essentially propagates through all calculations).
you might be able to grab sets directly from the smogon site. for example, as soon as you saw that heatran has fire blast, you could check the heatran analysis and assume that heatran is running one of the sets with that move. because there is no set that has both fire blast and flamethrower, the bot would then assume that the heatran doesn't have flamethrower.
That seems like it would be a good idea. I guess I'd have to learn how to do this first, though.
Anyway, I'm glad to see that the AI is going well :) I'm looking forward to incorporating it in my game engine (if I decide to use PokéLab or PO's battle system code, it shouldn't be too hard, right?).
For the most part, I don't think it really matters too much what battle system code you use. Right now the only simulator-specific code I have is a function that parses Poke Lab teams. However, I've separated that from the rest of my code, so I just need to duplicate that functionality for Pokemon Online teams, which is basically just me looking at the team format and sitting down for a few minutes and writing it. I plan on doing the same thing for the battle interaction. Have the program output use move slot 0, 1, 2, or 3, or switch to slot 0, 1, 2, 3, 4, 5, or 6 (or something along those lines), and then just have a function for each supported simulator that converts that to what they need. Should make things easy to port (I hope, anyway, I haven't looked into this at all, but letting it log on to a simulator is the next step after team prediction, which is cruising along).
Just an idea of how accurate this is, I was testing it out in #dreamworld the past two days, and I had two algorithms: cautious and aggressive. The aggressive algorithm seemed to have about 50% accuracy in guessing the team (it would generally get half of the unknown team members correct). However, in a good portion of teams, it would generally predict one member correct some of the time, and none correct some of the time. The cautious algorithm almost always get exactly one correct. However, I made a few updates to it last night that appear to have brought the two algorithms closer together, so hopefully each one will now have more of the strengths of the other. Here's sample output (bold == correct predictions):
AI said:
New team!
What Pokemon have you seen? Hippowdon
Cautious: Hippowdon, Blissey, Heatran, Tyranitar, Skarmory, Scizor
Aggressive: Hippowdon, Blissey, Skarmory, Tentacruel, Rotom-H, Swampert
What Pokemon have you seen? Skarmory
Cautious: Hippowdon, Skarmory, Blissey, Tentacruel, Tyranitar, Rotom-H
Aggressive: Hippowdon, Skarmory, Blissey, Tentacruel, Rotom-H, Swampert
What Pokemon have you seen? Tentacruel
Cautious: Hippowdon, Skarmory, Tentacruel, Blissey, Rotom-H, Spiritomb
Aggressive: Hippowdon, Skarmory, Tentacruel, Blissey, Rotom-H, Swampert
What Pokemon have you seen? Blissey
Cautious: Hippowdon, Skarmory, Tentacruel, Blissey, Rotom-H, Spiritomb
Aggressive: Hippowdon, Skarmory, Tentacruel, Blissey, Rotom-H, Swampert
What Pokemon have you seen? Celebi
Cautious: Hippowdon, Skarmory, Tentacruel, Blissey, Celebi, Rotom-H
Aggressive: Hippowdon, Skarmory, Tentacruel, Blissey, Celebi, Rotom-H
What Pokemon have you seen? Spiritomb
Cautious: Hippowdon, Skarmory, Tentacruel, Blissey, Celebi, Spiritomb
Aggressive: Hippowdon, Skarmory, Tentacruel, Blissey, Celebi, Spiritomb
However, order in which the Pokemon are seen is important. If it sees Hippowdon then Spiritomb, Aggressive gets all but Celebi correct (It predicts Rotom-H), and Cautious gets all but Celebi correct (it predicts Forretress).
One thing I've noticed about the Aggressive algorithm is that it always predicts a solid team. In other words, I've gone through the RMT forum, and where its predictions differ from the posted team, I generally prefer the AI's predicted team to the posted team.
Here are a few example teams where the bot only knows the lead Pokemon (bold Pokemon are the differences between the lists):
AI said:
Cautious: Azelf, Scizor, Heatran, Gyarados, Gengar, Flygon
Aggressive: Azelf, Scizor, Heatran, Flygon, Gyarados, Gengar
Cautious: Machamp, Heatran, Gyarados, Scizor, Gengar, Starmie
Aggressive: Machamp, Heatran, Gyarados, Flygon, Jirachi, Rotom-H
Cautious: Aerodactyl, Gengar, Scizor, Tyranitar, Gyarados, Starmie
Aggressive: Aerodactyl, Gengar, Tyranitar, Magnezone, Scizor, Starmie
Cautious: Metagross, Gyarados, Starmie, Tyranitar, Heatran, Gengar
Aggressive: Metagross, Gyarados, Heatran, Flygon, Scizor, Starmie
Cautious: Swampert, Scizor, Heatran, Gengar, Tyranitar, Dragonite
Aggressive: Swampert, Scizor, Heatran, Flygon, Rotom-H, Starmie
Cautious: Infernape, Scizor, Gengar, Gyarados, Starmie, Flygon
Aggressive: Infernape, Scizor, Starmie, Flygon, Gengar, Gyarados
Cautious: Heatran, Scizor, Flygon, Gyarados, Gengar, Starmie
Aggressive: Heatran, Scizor, Flygon, Gyarados, Gengar, Starmie
Cautious: Jirachi, Heatran, Gyarados, Tyranitar, Flygon, Gengar
Aggressive: Jirachi, Heatran, Flygon, Rotom-W, Suicune, Roserade
Cautious: Roserade, Heatran, Gyarados, Tyranitar, Rotom-H, Jirachi
Aggressive: Roserade, Heatran, Rotom-W, Flygon, Suicune, Jirachi
After my most recent update of including team mate stats that go both ways, Cautious and Aggressive have gotten a lot more similar than they were before.
Here's another example team that the AI can predict perfectly, my BP team. (Ninjask, Scizor, Vaporeon, Mr. Mime, Smeargle, Lucario):
AI said:
What Pokemon have you seen? Ninjask
Cautious: Ninjask, Scizor, Lucario, Vaporeon, Metagross, Breloom
Aggressive: Ninjask, Scizor, Starmie, Dragonite, Metagross, Gengar
What Pokemon have you seen? Scizor
Cautious: Ninjask, Scizor, Starmie, Vaporeon, Breloom, Lucario
Aggressive: Ninjask, Scizor, Starmie, Dragonite, Metagross, Gengar
And then any other Pokemon gives it the whole team correct with both algorithms. However, Scizor is the most common Pokemon on the team, and thus gives it the least information. The program seems to perform best when it's seen the rarest Pokemon. This is how it compares with any other member of the team (leaving off first step where I tell it Ninjask):
AI said:
What Pokemon have you seen? Lucario
Cautious: Ninjask, Lucario, Vaporeon, Scizor, Smeargle, Starmie
Aggressive: Ninjask, Lucario, Vaporeon, Smeargle, Mr. Mime, Scizor
AI said:
What Pokemon have you seen? Mr. Mime
Cautious: Ninjask, Mr. Mime, Smeargle, Vaporeon, Scizor, Lucario
Aggressive: Ninjask, Mr. Mime, Smeargle, Vaporeon, Lucario, Scizor
AI said:
What Pokemon have you seen? Smeargle
Cautious: Ninjask, Smeargle, Mr. Mime, Vaporeon, Scizor, Lucario
Aggressive: Ninjask, Smeargle, Mr. Mime, Vaporeon, Lucario, Scizor
AI said:
What Pokemon have you seen? Vaporeon
Cautious: Ninjask, Vaporeon, Mr. Mime, Smeargle, Lucario, Scizor
Aggressive: Ninjask, Vaporeon, Mr. Mime, Smeargle, Lucario, Scizor
For the Cautious algorithm, the earlier a Pokemon is listed, the more likely it is to be on the team. For the Aggressive algorithm, I believe the same is true. At the very least, earlier Pokemon are closer to 'reality', whereas later Pokemon are more 'predicted', because they're several degrees of statistics removed. The first predicted Pokemon for both algorithms will always be the same. Among revealed Pokemon, order doesn't matter, but which Pokemon on the team have been revealed is very important.
One more team prediction, on IRC, to show the weakness of the algorithm:
AI said:
[2010-11-22 08:57:07] <Optimusje> Gliscor
[2010-11-22 08:57:15] <davidstone> Cautious: Gliscor, Tyranitar, Heatran, Scizor, Vaporeon, Gyarados
[2010-11-22 08:57:15] <davidstone> Aggressive: Gliscor, Tyranitar, Scizor, Heatran, Gyarados, Rotom-H
[2010-11-22 08:57:29] <davidstone> Am I close?
[2010-11-22 08:57:56] <Optimusje> Not close, another one: kingdra
[2010-11-22 08:58:05] <davidstone> Cautious: Gliscor, Kingdra, Scizor, Heatran, Ludicolo, Tyranitar
[2010-11-22 08:58:05] <davidstone> Aggressive: Gliscor, Kingdra, Scizor, Ludicolo, Kabutops, Qwilfish
[2010-11-22 08:58:24] <Optimusje> Jirachi
[2010-11-22 08:58:31] <davidstone> Cautious: Gliscor, Kingdra, Jirachi, Heatran, Scizor, Tyranitar
[2010-11-22 08:58:31] <davidstone> Aggressive: Gliscor, Kingdra, Jirachi, Heatran, Rotom-W, Kabutops
[2010-11-22 08:58:48] <Optimusje> Infernape
[2010-11-22 08:58:56] <davidstone> Cautious: Gliscor, Kingdra, Jirachi, Infernape, Scizor, Tyranitar
[2010-11-22 08:58:56] <davidstone> Aggressive: Gliscor, Kingdra, Jirachi, Infernape, Scizor, Ludicolo
[2010-11-22 08:59:01] <davidstone> It's really having trouble with this team
[2010-11-22 08:59:02] <davidstone> =p
[2010-11-22 08:59:08] <Optimusje> Yeah lol
[2010-11-22 08:59:10] <Optimusje> Gengar
[2010-11-22 08:59:12] <davidstone> I've noticed every team with Kingdra on it that isnt' rain ruins it
[2010-11-22 08:59:23] <davidstone> Cautious: Gliscor, Kingdra, Jirachi, Infernape, Gengar, Scizor
[2010-11-22 08:59:23] <davidstone> Aggressive: Gliscor, Kingdra, Jirachi, Infernape, Gengar, Scizor
[2010-11-22 08:59:37] <Optimusje> Starmie is the last
[2010-11-22 08:59:40] <davidstone> Yikes!
[2010-11-22 08:59:52] <davidstone> Let me see if revealing the Pokemon in a different order would have made it more accurate
[2010-11-22 09:00:21] <davidstone> Yeah
[2010-11-22 09:00:29] <davidstone> If you had said Infernape second, this is its prediction at that stage:
[2010-11-22 09:00:34] <davidstone> Cautious: Gliscor, Infernape, Scizor, Tyranitar, Starmie, Gyarados
[2010-11-22 09:00:34] <davidstone> Aggressive: Gliscor, Infernape, Scizor, Tyranitar, Starmie, Gengar
[2010-11-22 09:00:43] <davidstone> It gets Starmie and Gengar with Aggressive
[2010-11-22 09:00:52] <Optimusje> Thats more like it yeah
[2010-11-22 09:01:01] <EeveeTrainer> davidstone i thought aggressive was the one least likely to get single pokemon right
[2010-11-22 09:01:17] <davidstone> It's more like on off the wall teams, Cautious is likely to get at least one
[2010-11-22 09:01:27] <davidstone> But when aggressive gets at least one, it usually gets several
[2010-11-22 09:01:38] <davidstone> It's more extreme in its predictions
[2010-11-22 09:01:52] <EeveeTrainer> davidstone if i have hippowdon on my team and it guesses shedinja
[2010-11-22 09:01:56] <davidstone> lol
[2010-11-22 09:01:57] <EeveeTrainer> youre banned from smogon forever
[2010-11-22 09:01:59] <davidstone> Funny story, actually
[2010-11-22 09:02:06] <davidstone> When I was testing it early, I put Shedinja as the lead
[2010-11-22 09:02:09] <davidstone> It predicted a Tyranitar
[2010-11-22 09:02:14] <davidstone> Because Shedinja isn't on the team mate stats
[2010-11-22 09:02:19] <davidstone> So it just predicted the top 5 Pokemon
[2010-11-22 09:02:23] <davidstone> lol
[2010-11-22 09:02:28] <davidstone> I wonder if it still does that
[2010-11-22 09:02:39] <davidstone> Nope
[2010-11-22 09:02:45] <davidstone> Cautious is usually dumb, like in this case
[2010-11-22 09:02:50] <davidstone> But I prefer the aggressive output, anyway
[2010-11-22 09:02:51] <davidstone> Cautious: Shedinja, Heatran, Scizor, Gyarados, Gengar, Tyranitar
[2010-11-22 09:02:51] <davidstone> Aggressive: Shedinja, Heatran, Scizor, Flygon, Gyarados, Gengar
In other words, final team is Gliscor, Kingdra, Jirachi, Infernape, Gengar, Starmie