It is better to have an approximate answer to the right question than an exact answer to the wrong one.
- John Tukey
About a year and a half ago, I had the opportunity to take an NLP class at NC State taught by Prof. Munindar Singh. The class was absolutely fascinating, and it led me to discover more about the classical side of NLP.
One of my favorite discussions in his class was about the Centering Theory. This framework from the 90s was used, among many things, to model coherence in a text. The idea is to start by dividing the text into utterances, or sentences for simplicity, and identifying the center, which would be the most salient part of an utterance (usually an entity). Once we have the center, we can identify how it moves in future utterances. If it bounces around too much, we can say that the text is less coherent. The second edition of Jurafsky and Martin's Speech and Language Processing book has a nice section on this topic, Section 18.1 on page 685.
We also had some other interesting discussions, such as about languages that are not written (e.g., Konkani), whether there are any true synonyms, how French has a more prominent gender classification, and how the French government tries to regulate their language.
Anyway, in his course, we were tasked with brainstorming and working on an NLP project. My colleague Anisha and I decided to work on reversing the twenty-questions game.
So, what is this game all about?
Some of you may have heard of the twenty-questions game or a similar online version called Akinator. The game involves a questioner and an answerer. The answerer thinks of a random person, such as a famous movie star like Tom Cruise. The objective of the game is for the questioner to guess this person in less than twenty questions by asking a series of yes-no type questions to the answerer. For instance, is the person male? Is he a movie star? Has he acted in any superhero films? And so on.
Today, efficient computer programs like Akinator can act as the questioner to try and guess the entity we are thinking about. They usually use space-partitioning methods to select questions (based on previously answered questions) that are likely to drastically decrease the search space.
However, we were trying to solve a reverse version of this problem:
Make the computer randomly choose a name and let the human guess the name by asking the computer a series of natural language questions. The game ends when the human correctly guesses the name selected by the computer.
In our research, we developed an entire NLP pipeline that included an entity formulation and pronoun resolution phase, a paragraph/sentence retrieval phase, and a boolean question-answering phase. Initially, we encountered some difficulty when working with completely random entities, so we decided to limit the domain to commonly known animals. Although our results were not incredibly promising, they were not underwhelming either.
If you are interested in our research, you can find our paper on Arxiv.org or view our slides.
Anyways, I had given up on this variant of the game until today I decided to play it with ChatGPT. To my surprise, the results were absolutely mind-boggling!
Here are three games I played with ChatGPT today. Although it failed the first game, it made accurate predictions in the second and third games. The italicized text represents ChatGPT's responses. This conversation is presented exactly as it occurred, without any edits .
Hey let us play a game of 20 questions where you think of someone in the
film industry and I will try and guess who that person is. Ready?
Sure, I'm ready! Go ahead and start.
Is it a male?
Yes, it is a male.
Is he young?
No, he is not young.
Has he recently acted in any blockbuster films?
No, he has not recently acted in any blockbuster films.
Has he ever acted in any blockbuster films?
Yes, he has acted in some blockbuster films, but he is not
necessarily known for them.
Has he acted in TV shows?
Yes, he has acted in TV shows as well.
Has he acted in a franchise film?
Yes, he has acted in a franchise film.
Does he act in action films?
Not typically, no. While he may have appeared in action films before,
he is not primarily known for acting in action films.
Does he act in drama films?
Yes, he is known for his roles in drama films.
Has he won an oscar?
Yes, he has won an Oscar for Best Actor.
Has he won an oscar multiple times?
No, he has won the Oscar for Best Actor only once.
Has any of his films made more than a billion dollars in box office
revenue?
No, none of his films have made more than a billion dollars in box
office revenue.
Is he above 60 in age?
Yes, he is above 60 in age.
Has he acted in gangster/mafia films?
Yes, he has acted in gangster/mafia films.
Has he acted in a martin Scorsese film?
Yes, he has acted in a Martin Scorsese film.
Was the film Goodfellas?
No, he did not appear in Goodfellas.
Has he acted in a film from Christopher Nolan?
Yes, he has acted in a film directed by Christopher Nolan.
I know the answer! Al Pacino!
I'm sorry, but the person I was thinking of is not Al Pacino. Do you
want to continue asking questions or make another guess?
I want to continue guessing, dont tell the answer
Okay, feel free to ask another question!
Was he in True Detectives?
No, he did not appear in True Detective.
Has he acted in Titanic?
No, he did not appear in Titanic.
Okay! I give up! Tell me the answer?
The person I was thinking of is Anthony Hopkins. He is a Welsh actor
known for his work in films such as The Silence of the Lambs (for
which he won an Oscar), The Remains of the Day, and The Two Popes,
among others. He has also appeared in TV shows like Westworld and
Frasier, and in films directed by Christopher Nolan (The Prestige) and
Martin Scorsese (The Edge of Darkness).
Wait what? Anthony Hopkins was not in The Prestige!
I apologize for the mistake. You are correct, Anthony Hopkins did not
appear in The Prestige. That was my error.
Uhh! That was why I was unable to guess
I'm sorry about that. I made a mistake, which probably made the
guessing game more difficult. If you want to play another round of the
game, I'd be happy to participate and try to do better this time!
As you can see, there are quite a few issues with this conversation. It's worth noting that Martin Scorsese did not direct The Edge of Darkness ; that was actually Martin Campbell. Additionally, it's important to point out that Anthony Hopkins did not appear in The Prestige or Frasier.
Okay lets play another round
Sure! Go ahead and start with your first question.
Is it a male actor?
Yes, it is a male actor.
Is he older than 50?
No, he is not older than 50.
Is he younger than 30?
No, he is not younger than 30.
Has he acted in a blockbuster franchise?
Yes, he has acted in a blockbuster franchise.
Has the franchise grossed over a billion dollars in box office
revenue?
Yes, the franchise he acted in has grossed over a billion dollars in
box office revenue.
Was the franchise related to superhero genre?
Yes, the franchise he acted in is related to the superhero genre.
Was it MCU?
Yes, the franchise he acted in is a part of the Marvel Cinematic
Universe (MCU).
Has he acted in The Avengers movie?
Yes, he has acted in The Avengers movie.
Was he one of the Avengers?
Yes, he was one of the Avengers.
Has he acted in Sherlock Holmes franchise?
No, he has not acted in the Sherlock Holmes franchise.
Has he acted in Knives Out?
No, he did not appear in Knives Out.
Was he part of Ghostbusters reboot?
No, he was not part of the Ghostbusters reboot.
Did he act in a TV show from MCU?
Yes, he has acted in a TV show from the MCU.
Has he played the Hulk?
No, he did not play the Hulk.
Does he throw arrows in the Avengers film?
Yes, he throws arrows in The Avengers film.
Jeremy Renner?
Yes, you are correct! The actor I was thinking of is Jeremy Renner.
He is known for his role as Clint Barton/Hawkeye in the Marvel
Cinematic Universe, including The Avengers movie, as well as other
films like The Hurt Locker and Arrival. He has also appeared in TV
shows like The Unusuals and most recently in the Hawkeye TV series.
Good One!
Thank you! It was fun playing the game with you again. Let me know if
you want to play another round or if you have any other questions.
I found it incredibly fun! Can we play another game?
Absolutely! I'm ready whenever you are. What would you like me to
think of this time?
Same, actor/actress in the film industry
Sure thing! I'm thinking of someone in the film industry again. Let's
begin!
Is it male?
No, it is not male.
Has she acted in any blockbuster films with a franchise?
Yes, she has acted in at least one blockbuster film with a
franchise.
Has that film grossed over a billion dollars in box office
collection?
Yes, the film she acted in has grossed over a billion dollars in box
office revenue.
Are we talking about the MCU franchise?
No, I'm not thinking of an actress from the Marvel Cinematic
Universe.
Has she acted in superhero films?
Yes, she has acted in superhero films.
Is she from the DC franchise?
Yes, she is from the DC franchise.
Is she from DCEU?
Yes, she is from the DC Extended Universe (DCEU).
Did she act in the Justice League?
Yes, she did.
Does her character in that film wear a red suit?
No, her character in that film does not wear a red suit.
Is she a reporter in that film?
Yes, her character is a reporter in the film.
Is she related to Superman in the film?
Yes, she is related to Superman in the film.
Has she acted in Arrival and the TV show The Office?
Yes, the actress is Amy Adams.
Gotcha! :)
Glad you enjoyed the game! Do you want to play another round or is
there anything else I can assist you with?
That's it, folks! I hope you enjoyed reading this blog post as much as I enjoyed writing it. With ChatGPT's remarkable capabilities, it's clear that much of the NLP research of the past will be superseded. And this is only the beginning! I'm excited to see what the future holds for this technology.
In the meantime, I encourage you to explore this game on your own and share it with friends and family. If you have any questions or comments, please don't hesitate to reach out to me. I'd love to hear from you!