Rick Rosner On, 15: AI, IQ Tests, the Mega Society, and Machine Intelligence

Author(s): Scott Douglas Jacobsen

Publication (Outlet/Website): Personal SubStack

Publication Date (yyyy/mm/dd): 2026/07/17

Scott Douglas Jacobsen interviews Rick Rosner about Mega Society members testing artificial intelligence on difficult IQ-style problems. Rosner says AI increasingly excels at number series through speed, training data, and repeated attempts, yet remains vulnerable to visual patterns, ambiguous rules, and “dirty” problems requiring real-world knowledge. They also examine intelligence testing’s educational and legal purposes, its psychometric limits, declining unaided analytical habits, and whether assessments should measure human competence while using phones and AI tools.

Scott Douglas Jacobsen: What is new with the Mega Society and the Noesis guys?

Rick Rosner: I check in with them often. I have been talking back and forth with some people there. They are working with Fable and other AIs to see how good they are at solving IQ-type problems, hard IQ test-type problems. Alternatively, conversely, to see whether they can come up with IQ-type problems that AI has trouble with. I have got to say that I am one of the best solvers of IQ problems in the world, but on many of these problems, AI has me beat now.

Jacobsen: What are they defining as an IQ-type problem? Can you run through that more precisely?

Rosner: The focus, from what I have seen, and I do not know everything they are doing, is number series. The AIs are pretty good with number series. They have been trained on massive amounts of data. If a number series has been published somewhere and was included in their training data, the model may already have encountered it, which is highly helpful. Sometimes it takes them a few tries. AI can confidently be wrong. Even when it comes to math, it will make bad errors, saying, “This is how the series works. You add 45 to this, then you double 45. You add 90 to this, then 180.” Then the actual addition is wrong. You add 45 to 23 and get 150. No, you do not. AI will do that. However, in terms of speed, you can tell AI, “No, you are wrong. Try again.”

It might take me 20 hours of dicking around with a tough problem. On some of the crazy-hard problems, it might take more than that. AI can quickly go through possible approaches. Does AI have the full quiver of techniques a human might have? No, not yet. However, it is effective in the techniques it does have and in its ability to draw on vast amounts of information. Imagine going on a quiz show with an enormous library in your head and being able to search it within seconds. You would do well on that quiz show.

Jacobsen: There are differences between the types of problems artificial intelligence can handle. AI had many successful forms before large language models, but LLMs became especially prominent for their performance in language. Different systems have different architectures, training methods, datasets, iterations, and development timelines. So, their capabilities differ.

Text-based problems are often a strength for language models. Verbal problems may be among the ones they solve most readily, although that does not mean they will solve every difficult verbal problem. The Mega and Titan tests contain verbal, numerical, and spatial problems, so they are not simply verbal tests. Mathematical reasoning has also become a major area of AI development. Spatial reasoning is being studied through multimodal systems, while protein-folding systems such as AlphaFold are specialized models rather than evidence that ordinary LLMs possess general spatial intelligence. How does that apply to these fairly standard categories?

Rosner: The Noesis people have sent me some printouts of AI reports explaining why the systems believe they have solved various problems. From reading these reports, I am not familiar with all sorts of jargon, along with various AI models and iterations. I’m not sure whether they are all LLMs. I’m not aware of the other types they are using. The Noesis guys can work with the AIs to select or configure models with specialties relevant to the problems they are working on.

Text-only LLMs have trouble with visual stuff because they do not directly process images. Multimodal models can process images, but they can still have trouble with visual and spatial reasoning. They cannot necessarily reliably work out angles or geometric relationships. We have talked a lot about one of Koyman’s requisites for being good at solving IQ problems: associative width or breadth. AI systems can have a broad associative reach, but they do not necessarily approach a problem visually as a human solver does.

If you give an AI a hint by saying, “You might have problems with this because the solution requires being able to see,” that might be a big enough clue to point it in the right direction. Even if a text-only AI cannot see the symbols directly, it can reason from a verbal description of their visual characteristics. A multimodal model can inspect the image itself, though that still does not guarantee that it will understand the pattern correctly.

You could put together an elementary series using numbers with holes in them. A series might be 0, 4, 6, 9, 10, 14, 16, 19, and so on, as long as each number contains a closed loop when written in a particular typeface. AI would not necessarily solve that series easily unless you gave it a hint. What I said might apply even to that simple series.

As has been the case since public versions of different AI models became available, the speed at which they seem to be progressing remains shocking, at least to people like me who are not versed in AI at the nuts-and-bolts level.

Jacobsen: Who on the Mega Society team is working on this stuff, who has nuts-and-bolts experience?

Rosner: I do not know. The two guys I know are Chris Cole and Dean Inada. There is also someone named Warner who appears in many of the emails. I do not know whether I know him personally.

Chris and Dean are good at working with this stuff. I expect their knowledge of it is more fully developed than mine in many areas. I have gone the word route rather than the technological or mathematical route. My discipline for learning math and physics has been shitty. I have written millions of words.

I believe these guys have a very strong understanding of what is going on when AI does whatever the fuck it does.

Jacobsen: Earlier in the conversation, you referred to certain areas in which they find that AI has limitations with IQ-type problems. What are those limitations?

Rosner: The limitations are more pronounced when it comes to creating problems. Let me finish this point, and then I will get to the larger point.

Given the size of the datasets on which AIs are trained, AIs might still have difficulty solving certain types of numerical problems. I call at least some of the number series that AI has trouble with “dirty series” because they step outside the realm of pure numbers and into what Chris Cole might call trivia.

There was a well-known IQ-type problem from decades ago that went something like 72, 79, 86, 96, and so on. The answer depended on recognizing the sequence as subway stops along a particular New York City line, rather than deriving it from a purely mathematical rule. You had to recognize that an apparently arbitrary string of numbers referred to something in the real world.

I consider that a dirty problem because it requires you to know some real-world trivia to figure out the pattern. You can defeat AI by stepping outside the expected domain. But it is hard to devise a pure number-series problem that exploits a principle of numbers without going outside the realm of mathematics, and that a capable AI cannot eventually solve.

Then you get into the larger issue of IQ testing. IQ testing is useful for the purposes for which early intelligence tests were developed more than a century ago by Alfred Binet and Théodore Simon. You test children in school to estimate their cognitive development relative to their peers and determine whether they may need specialized educational support.

Enrichment or additional resources in school. That is a very legitimate use of IQ testing. I suppose you could also use it with adults, such as somebody on death row or somebody on trial for a capital crime. You give that person an IQ test, and the score comes out around 65.

Then the defence can say, “This person is accused of committing these murders, but with an IQ of 65, what is the extent of their intellectual disability? How well can they understand right from wrong, assist counsel, or grasp the consequences of their actions? Is it constitutional or fair to execute somebody with substantial intellectual limitations?”

That issue comes up in death penalty cases. In the United States, the Supreme Court has ruled that executing a person with an intellectual disability violates the Eighth Amendment. But an IQ score alone does not settle the issue. Courts are also supposed to consider adaptive functioning and the person’s developmental history. A score of 70 or 75 can be relevant, given the test’s margin of error, but it is not an automatic legal cutoff.

That is another legitimate, fairly conventional use of IQ testing, as opposed to the sporting use of IQ, which is, “Let us see whether we can design a test with a ceiling of 200.”

Jacobsen: Binet found that intelligence develops unevenly with age and that complex abilities, such as judgment, problem-solving, and attention, cannot be measured accurately through simple sensory or reaction-time tests. He and Théodore Simon developed an early scale intended to identify children who might need educational support. The latter concept of mental age compares a child’s performance with the typical performance of children at different chronological ages.

What is especially important, and contrary to some later uses of intelligence testing, is that Binet resisted treating intelligence as a single fixed or purely innate quantity.

Rosner: Look at somebody like Trump, whose mental functioning is often called into question. People sometimes ask, “What do you think Trump’s IQ is?” But the more pertinent question is, “How intelligently does Trump function?” He is not likely to take and publicly release the results of a comprehensive IQ test.

He has taken brief cognitive screening tests, including the Montreal Cognitive Assessment, and has claimed perfect scores. That is not an IQ test. It is designed primarily to screen for possible cognitive impairment, not to measure the full range of intelligence. His physician reported another perfect score in 2026, although public debate about the completeness and transparency of presidential health disclosures has continued.

A further problem with Trump is that he lies and bullshits a lot. He appears mentally lazy. His con-man personality may effectively lower the quality of his thinking because he has bullshitted his way through much of his life. He seems to have a short attention span, limited curiosity, and little patience for reading more than a paragraph.

Some of what is going on with Trump is happening with everybody now because our reading habits and attention spans are changing as we receive information. It is more severe with fucking Trump, but the issue is not simply his underlying intelligence. It is also his apparent refusal to work hard at using whatever intelligence he has.

To get to the larger point, what does it mean to test raw intelligence? You sit a child, or a classroom full of children, down for 45 minutes, give them a bunch of problems, and do not let them use any tools except their brains and a pencil. That tells you something, but it does not tell you everything.

When researchers compare test scores across generations, they have found evidence in several countries that the long-running Flynn effect has slowed, stalled, or reversed on some measures. That does not establish that people are simply “getting dumber.” The pattern varies across countries, age groups, tests, and cognitive domains, and the causes remain debated. Environmental changes, education, test familiarity, health, technology, and other factors may all matter.

Still, there is a legitimate concern that people may be losing some analytical skills when they are forced to think without recourse to their devices. That creates huge educational challenges. High schools, middle schools, and colleges are struggling with declining attention spans, uneven preparation, and the widespread use of AI to do work students are supposed to do themselves.

But it also makes you wonder whether we should test people on how well they function with a fucking phone in their hands, because that is how people function in the real world now. If somebody appears to be a fucking idiot when you take the phone away but is perfectly competent with it in hand, then perhaps the phone has become part of the person’s effective cognitive equipment.

Should we not be developing some testing program that measures people’s competence when working with devices?

Jacobsen: Thank you very much for the opportunity and your time, Rick.

Last updated May 3, 2025. These terms govern all In-Sight Publishing content—past, present, and future—and supersede any prior notices.  In-Sight Publishing by Scott  Douglas  Jacobsen is licensed under a Creative Commons BY‑NC‑ND 4.0; © In-Sight Publishing by Scott  Douglas  Jacobsen 2012–Present. All trademarks, performances, databases & branding are owned by their rights holders; no use without permission. Unauthorized copying, modification, framing or public communication is prohibited. External links are not endorsed. Cookies & tracking require consent, and data processing complies with PIPEDA & GDPR; no data from children < 13 (COPPA). Content meets WCAG 2.1 AA under the Accessible Canada Act & is preserved in open archival formats with backups. Excerpts & links require full credit & hyperlink; limited quoting under fair-dealing & fair-use. All content is informational; no liability for errors or omissions: Feedback welcome, and verified errors corrected promptly. For permissions or DMCA notices, email: scott.jacobsen2025@gmail.com. Site use is governed by BC laws; content is “as‑is,” liability limited, users indemnify us; moral, performers’ & database sui generis rights reserved.