AI is wooly animal stripped of aces (3)
How do we measure sentience, intelligence and ultimately humanness? AI, or more accurately Language Learning Models (LLMs), can pass the Imitation Game, the test devised by Alan Turing to measure human intelligence in machines, but we know they’re not exhibiting true intelligence. So, how can we measure a machine’s intelligence? Enter… the Cryptic Crossword Test (trademark pending*).
For all the amazing things we are seeing Generative AI create, it can’t solve a cryptic crossword (yet). For anyone not familiar with a cryptic crossword, it’s when the clue itself is a puzzle that needs cracking.
Here’s an example of a very typical style of cryptic crossword clue:
Former lover can court no more (7)
Things to look out for when solving:
Look for possible definitions – usually at the beginning or end of the clue.
Look for synonyms within the clue e.g. could ‘former lover’ be written as something else?
It’s not usually what it sounds like it should be i.e. the answer probably isn’t about courting former lovers.
There’s a good chance the question setter is trying to have fun with you.
The answer… EXTINCT (former lover – also called an ‘ex’, can – another word for ‘tin’, court – abbreviated to CT, think roads signs, no more – the definition).
What does AI make of this clue:
The jump from ‘former lover’ to meaning dead/deceased is an interesting stretch. But the big red flag here is the fundamental rule of crosswords, it needs to be the right number of letters. ‘Deceased’ is an eight letter word.
So why can’t cryptic crosswords be solved by LLMs? Yes, they can pass the Imitation Game/Turing Test, but the Cryptic Crossword Test reveals the complete absence of intelligence and humanness in LLMs.
The skills required to solve a cryptic crossword match those required to generate consumer insights. If you haven’t ever attempted to solve one, you might be unconvinced… here’s our thinking.
CONTEXTUAL KNOWLEDGE
Cryptic clues often rely on cultural references, specialised vocabulary, and contextual knowledge that AI lacks. Whether working in one market or cross-market, understanding the consumer backdrop is essential to understanding your consumer.
WORDPLAY AND DOUBLE MEANINGS
Cryptic clues intentionally use ambiguous or misleading wording. Successfully solving them relies heavily on picking up nuances in language. The answers to the research questions are hidden within the words consumers use, they’re not directly given to us.
REASONING AND INFERENCE
Solving cryptics requires making logical leaps and reading between the lines. The reasoning involved goes beyond pattern recognition. This is what turns a good research project into a great one.
INTUITION
Expert cryptic solvers develop an intuitive skill for solving cryptic clues. AI lacks intuition that comes from our lived experiences. Those that make us human and give us the ability to take that ‘gut-feel’ or ‘hunch’ into consumer testing.
And if we get to the point that AI is successfully solving cryptic crossword puzzles, you’ll find me on a desert island, hiding from technology!
Have you managed to solve the puzzle in the title? LLMs like ChatGPT aren’t going to be able to help you.
Hint: think of a woolly animal and remove the ‘a’s.
Have we convinced you of your value as a (human) researcher? Or imbued a new passion for cryptic crosswords? We’d love to know.
*trademark not pending
If you’d like to read more on our thoughts on AI, we’ve also considered whether AI could get a job at Magenta.