The New York Times have posted the game Rock-Paper-Scissors – You vs. the Computer in the wake of the big Watson computer win over the human Jeopardy contestants. They reduce AI to “building on simple rules and statistical averages” and present a game of chance, with some big claims.
They have two versions of the game, one that has to learn as it plays, the other that has some experience in the mix.
Picking an Opponent
I’m no rookie, I have been known to dominate the RPS scene at my office and I know how to stare down an opponent. Oh, wait…shoot. I start with the sure-fire winner, the one you should put your money on.
The Veteran Presence
So here’s their idea…after several rounds of play their computer will use statistical analysis to figure out your strategy, backed by 200,000 previous rounds of play. They don’t define the 200,000 rounds – as in, were they human played, were they 200,000 different people, if nationality or race or education played in – but for some reason that’s enough data to “exploit a person’s tendencies”, and yet, I have my doubts.
I decided to play 40 rounds. If there was anything akin to AI in the pot, this thing should at least win one third of its games.
Here’s where I was a few moves in:
While it made some ground up towards the end, I was able to hold it off of any streak and the closest it got was within two.
After that, I teetered on +2-3 points and as high at +4-5 points, and ended up +4 over the computer, thus solidly proving that, minimally, if we’re going to partner a game of chance with technology called “AI”, we have a long way to go.
There was no stopping me by some ‘simple rules’ engine. ;o)
I want to point something important out about this statistic: for the one who is winning more that the opponent and the tie, they are beating the odds. I am a combined +10 on 40 rounds here at the end of the game, meaning that not only am I beating an expert player, but I’m beating the tie-chance handily as well.
Bring in the Rookie
Finally, I tried out the “dumb” version, the rookie alter-ego that must first learn the rules and doesn’t get the benefit of 200,000 other players’ moves in a database. I decided to give it an extra 5 moves as a chance to learn some of the rules before getting into the play.
What? Shocker?! How could it be that with 5 more moves against an (arguably) weaker opponent I could have fewer wins?!
Oh right…it a freeking game of chance. Unless you put on your idiot cap and play the same hand each time, this is a game of odds and they are 1/3, 1/3, 1/3 as to what I’ll play next. Period. Just because 200,000 other people picked R-R-P-S-S-R before me doesn’t mean that I’ll play P instead of R on my next move.
This is not a prediction, even when backed by hundreds of thousands of records, not any more than a lady in a tent with a lit glass ball. Over time I will win 33%, lose 33% and tie 33%. It’s no different than a coin toss, with one more side to the coin.
The only learning that this ‘AI’ does is around the rules of the game. To that end, there’s only really six (counting ties), and it seems to know when it’s learned them all, suggesting a bias in the first place.
Okay, Yeah, But…
There’s more going on. I know it does some pattern matching and what not, but that doesn’t mean it’s AI.
After playing these games I walked through it’s demonstration about how it thinks. It uses some good buzz words making it sound like it’s thinking, but really, this is just straightforward math equations muxed with some simple filtering that any first year CompSci student should be able to figure out.
No magic here, folks, move along.
Conclusion
The programmers at the New York Times have created a good gimmick that will fool most of the general population into believing that they have created some form of artificial intelligence that, when it mates with Watson, is ready to techno-freak the humans into submission.
Hey, it may even be an impressive chunk of code, but it does not represent any of the modern complexities that some impressive minds (like those at IBM) have solved.
I could likely to write a version just as capable by picking randomly the answer.
Better yet, I’m going to write a traffic light color prediction engine.
Look out, Matrix, we’re all about to take the blue pill. Or was that the red?
No comments:
Post a Comment