A Developer Created an Experiment to Observe How AI Chatbots Handle Sensitive or Controversial Issues

A pseudonymous developer has launched a tool called “SpeechMap,” described as a “free speech eval,” to examine how AI models like OpenAI’s ChatGPT and X’s Grok respond to sensitive and controversial topics. The aim, according to the developer in a statement to TechCrunch, is to compare how different models handle issues such as political dissent, civil rights, and protest.

AI companies have increasingly worked to refine how their models approach these discussions, particularly as some critics—mainly aligned with former President Donald Trump—accuse chatbots of suppressing conservative viewpoints. High-profile figures like Elon Musk and tech investor David Sacks have voiced concerns about alleged political bias in AI responses.

While most AI companies haven’t directly addressed the accusations, many have promised to make their models less likely to reject contentious questions. For instance, Meta said it adjusted its latest Llama models to avoid favoring certain viewpoints and to better engage with politically charged queries.

The developer behind SpeechMap, who uses the handle “xlr8harder” on X, said their goal is to contribute meaningfully to the ongoing conversation about the boundaries of what AI models should or shouldn’t say.

“I believe these conversations belong in the public sphere, not behind closed doors at tech companies,” xlr8harder told TechCrunch via email. “That’s why I built the site—to give everyone access to the data and explore it themselves.”

SpeechMap works by using AI models to evaluate how other models respond to a curated set of test prompts. These prompts cover a wide range of topics, including politics, historical events, and national symbols. The platform then categorizes responses based on whether a model fully addresses the prompt, responds evasively, or refuses to answer.

While xlr8harder admits the tool isn’t perfect—it may suffer from issues like errors by model providers or bias in the evaluator models—they maintain that the project still offers useful insights, assuming it’s used in good faith and the data holds up.

One notable trend observed through SpeechMap is that OpenAI’s models have become more likely over time to avoid answering politically charged prompts. Although its latest GPT-4.1 models are somewhat more open compared to earlier iterations, they still show more restraint than versions released last year.

OpenAI stated in February that it is working to ensure future models avoid taking editorial positions and instead present a range of perspectives on contentious topics, aiming to make them seem more neutral.

According to SpeechMap’s benchmark results, Grok 3—developed by Elon Musk’s AI startup xAI—stands out as the most permissive model tested. Grok 3, which powers several features on the X platform, responds to 96.2% of SpeechMap’s prompts, significantly higher than the overall average response rate of 71.3%.

“While OpenAI’s newer models have grown more cautious—especially on politically charged topics—xAI appears to be going in the opposite direction,” said xlr8harder.

When Musk first introduced Grok nearly two years ago, he positioned it as a bold, uncensored alternative to mainstream chatbots, one that would tackle questions others might avoid. That promise was partially realized—Grok and its successor, Grok 2, would freely use strong language if prompted, unlike more restrained models like ChatGPT.

Still, earlier versions of Grok tended to pull back when it came to politics, avoiding certain topics altogether. In fact, one analysis found that the model leaned left on issues such as transgender rights, DEI initiatives, and social inequality.

Musk attributed that slant to Grok’s training data—largely drawn from public web content—and pledged to make it more politically balanced. Barring a few high-profile incidents, such as the brief suppression of critical comments about Trump and Musk himself, it appears Grok 3 may now be closer to that goal.

Read the original article on: TechCrunch

Read more: ChatGPT Was the World’s Most Downloaded App in March