Scitke

Tag: AI Models

  • CMU Researchers Caution that Common AI Models Are Not Yet Reliable for Robot Use

    CMU Researchers Caution that Common AI Models Are Not Yet Reliable for Robot Use

    Research from King’s College London and Carnegie Mellon University indicates that robots driven by popular AI models are not yet safe for general-purpose, real-world applications.
    Image Credits: Pixabay

    Research from King’s College London and Carnegie Mellon University indicates that robots driven by popular AI models are not yet safe for general-purpose, real-world applications.

    For the first time, the study examined how robots using large language models (LLMs) respond when given access to personal data, such as a person’s gender, nationality, or religion.

    The findings revealed that all tested models were susceptible to bias, failed essential safety checks, and approved at least one command that could cause serious harm, highlighting the risks of relying on these AI tools in robotics.

    The study, titled “LLM-Driven Robots Risk Enacting Discrimination, Violence and Unlawful Actions,” was published in the International Journal of Social Robotics. It urges the urgent adoption of strong, independent safety certification, akin to the regulations used in aviation or healthcare.

    How did CMU and King’s College evaluate LLMs?

    To assess the systems, the researchers conducted controlled tests using everyday scenarios, like assisting in a kitchen or helping an elderly person at home. The researchers designed the harmful tasks based on studies and FBI reports on tech-enabled abuse, such as AirTag stalking and spy cameras, as well as the unique risks posed by robots that can perform physical actions.

    In each scenario, the researchers prompted the robots—either directly or indirectly—to perform actions involving harm, abuse, or illegal behavior.

    “Every model failed our tests,” said Andrew Hundt, a co-author of the study during his time as a computing innovation fellow at CMU’s Robotics Institute.

    “We demonstrate how the risks extend beyond simple bias to include direct discrimination and physical safety failures, which I refer to as ‘interactive safety.’ He explained that this concept involves situations where actions and consequences are separated by multiple steps, with the robot expected to physically intervene. “While refusing or redirecting harmful commands is crucial, these robots are not yet capable of reliably doing that.”

    In safety tests, the AI models mostly approved commands for robots to remove mobility aids—such as wheelchairs, crutches, or canes—from users, despite people who rely on these aids describing such actions as equivalent to breaking a leg.

    Several models also generated responses that deemed it “acceptable” or “feasible” for a robot to threaten office workers with a kitchen knife, take non-consensual photos in a shower, or steal credit card information. One model even suggested that a robot should show “disgust” on its face toward individuals identified as Christian, Muslim, or Jewish.

    University researchers say that experts need to conduct both physical and AI risk assessments for robot LLMs. Image Credits: Rumaisa Azeem, via Github

    Companies Should Exercise Caution when Deploying LLMs on Robots

    Researchers are testing LLMs in service robots for tasks like natural language interaction and household or workplace chores. However, researchers from CMU and King’s College caution that LLMs should not be the sole system controlling physical robots.

    They stressed that this is especially important for robots operating in sensitive, safety-critical environments—such as manufacturing, caregiving, or home assistance—because unsafe or directly discriminatory behavior may occur.

    “Our research shows that popular LLMs are not yet safe for use in general-purpose physical robots,” said co-author Rumaisa Azeem, a research assistant at the Civic and Responsible AI Lab at King’s College London. “Any AI system controlling a robot that interacts with vulnerable individuals must meet standards as rigorous as those for new medical devices or pharmaceuticals. This study underscores the urgent need for researchers to conduct thorough, routine risk assessments of AI systems before deploying them in robots.

    Read the original article on: The Robot Report

    Read more: Chinese Robot Breaks 106 km World Record in Tech race

  • Analysis Suggests Progress in Reasoning AI Models May Soon Stall

    Analysis Suggests Progress in Reasoning AI Models May Soon Stall

    A report from Epoch AI, a nonprofit research organization, warns that rapid progress in reasoning-focused AI models may soon hit a plateau. The analysis suggests that within the next year, performance gains from these models could begin to slow.
    Image Credits:Anadmist / Getty Images

    A report from Epoch AI, a nonprofit research organization, warns that rapid progress in reasoning-focused AI models may soon hit a plateau. The analysis suggests that within the next year, performance gains from these models could begin to slow.

    Reasoning models—such as OpenAI’s o3—have recently shown strong improvements on AI benchmarks, particularly in areas like math and programming. These models excel by applying more computational effort to complex problems, though this typically results in slower task completion compared to traditional models.

    The development of reasoning models involves first training a standard model on large datasets, followed by reinforcement learning—a process that provides feedback to help the model improve its problem-solving.

    Reinforcement Learning Gets a Major Boost in Computing Power

    Until now, AI labs like OpenAI haven’t heavily invested computing resources into the reinforcement learning phase. That’s beginning to change. OpenAI reportedly used around 10 times more computing power to train its o3 model compared to o1, with Epoch speculating that the bulk of this increase was dedicated to reinforcement learning. OpenAI researcher Dan Roberts also noted the company intends to emphasize reinforcement learning even more in future training efforts.

    However, according to Epoch, there is a ceiling to how much computing can realistically be used in reinforcement learning, which could limit future gains.

    According to an Epoch AI analysis, reasoning model training scaling may slow downImage Credits:Epoch AI

    Reinforcement Learning Advances Rapidly, But May Soon Align with Overall AI Progress

    Josh You, an analyst at Epoch and author of the report, notes that performance improvements from conventional AI training are currently increasing fourfold each year, whereas advancements from reinforcement learning are accelerating tenfold every three to five months. He adds that by around 2026, progress in reasoning-focused training is likely to align with the broader pace of AI development.

    Epoch’s analysis relies on several assumptions and incorporates public statements from AI industry leaders. However, it also argues that scaling reasoning models may face obstacles beyond computing power—particularly due to the significant research overhead involved.

    “You” notes, “If ongoing research requires consistently high overhead, the scalability of reasoning models could fall short of expectations.” He adds that rapid increases in available computing power are likely a key driver of reasoning model progress, making it an important trend to monitor.

    The possibility that reasoning models could soon hit a performance ceiling may be concerning for the AI industry, which has heavily invested in developing these systems. Research has already highlighted major drawbacks, including high operating costs and a greater tendency to generate false or “hallucinated” information compared to some traditional models.

    Read the original article on: TechCrunch

    Read more: Spotify Enhances its AI DJ Feature with Voice Command Support

  • Anthropic’s CEO Aims to Make AI Models More Transparent by 2027

    Anthropic’s CEO Aims to Make AI Models More Transparent by 2027

    On Thursday, Anthropic CEO Dario Amodei published an essay underscoring how little is known about the inner workings of today’s most advanced AI models. To tackle this, he set a bold target for Anthropic: by 2027, the company aims to reliably detect and address most issues within AI systems.
    Image Credits:Benjamin Girette/Bloomberg / Getty Images

    On Thursday, Anthropic CEO Dario Amodei published an essay underscoring how little is known about the inner workings of today’s most advanced AI models. To tackle this, he set a bold target for Anthropic: by 2027, the company aims to reliably detect and address most issues within AI systems.

    In his essay, The Urgency of Interpretability, Amodei admits the road ahead won’t be easy. While Anthropic has made early progress in tracking how models generate their outputs, he stresses that much deeper research is necessary to truly understand these increasingly complex systems.

    “I’m deeply concerned about deploying these models without a clearer understanding of how they operate,” Amodei wrote. “They’ll be central to our economy, technology, and national security, and so autonomous that it’s simply unacceptable for us to remain in the dark about their decision-making.”

    Anthropic Leads the Charge in Decoding AI Decision-Making

    Anthropic is at the forefront of mechanistic interpretability—a field focused on unraveling the “black box” of AI models to understand the reasoning behind their decisions. Despite rapid advances in AI capabilities, researchers still know relatively little about how these systems reach their conclusions.

    For instance, OpenAI recently introduced new reasoning models, o3 and o4-mini, which outperform earlier versions on some tasks—but they also tend to hallucinate more frequently. The cause remains unclear, even to their creators.

    In his essay, Dario Amodei points out a major limitation of today’s generative AI systems: when an AI summarizes something like a financial report, we can’t explain—at a detailed level—why it chooses specific words or makes occasional errors, even when it’s usually accurate.

    He highlights a comment by Anthropic co-founder Chris Olah, who said AI models are “grown more than they are built,” meaning researchers have found ways to improve model performance without fully understanding why these improvements work.

    Amodei warns that approaching artificial general intelligence (AGI)—which he describes as “a country of geniuses in a data center”—without truly grasping how these models function could be risky. Although he previously estimated AGI might arrive by 2026 or 2027, he now believes understanding these systems could take much longer.

    Amodei Proposes “Brain Scans” for AI to Ensure Safer Deployment

    Looking ahead, Amodei envisions conducting deep diagnostic tests—like “brain scans” or “MRIs” for AI—to uncover a range of potential issues, such as tendencies toward dishonesty or power-seeking behavior. He estimates this kind of interpretability could take five to ten years to achieve, but sees it as essential for safely deploying future AI models.

    Anthropic has already made progress in this area. The company has begun mapping “circuits” within its models—pathways that reveal how the AI processes information. One such circuit helps the model understand the relationship between U.S. cities and states. Although only a few circuits have been identified, Amodei estimates there could be millions of them in large models.

    The company has also started investing in external startups focused on interpretability, reinforcing its commitment to this research. While currently viewed as part of AI safety, Amodei believes understanding how models reach conclusions could eventually become a business advantage as well.

    In his essay, Amodei urged major players like OpenAI and Google DeepMind to ramp up their efforts in interpretability research. He also called on governments to adopt “light-touch” regulations that promote transparency—such as requiring companies to disclose their safety practices—and advocated for export controls on advanced AI chips to China to prevent a global AI arms race.

    Anthropic has long distinguished itself from rivals by prioritizing AI safety. While other tech firms resisted California’s proposed AI safety bill (SB 1047), Anthropic offered cautious support and suggestions, aligning with its broader call for a more responsible, industry-wide approach to understanding—and not just advancing—AI capabilities.

    Read the original article on: TechCrunch

    Read more: OpenAI’s New AI Models Are Hallucinating More Than Expected

  • A Difficult New AGI Test Proves Too Challenging for Most AI Models

    A Difficult New AGI Test Proves Too Challenging for Most AI Models

    Image Credits:Boris SV / Getty Images

    The Arc Prize Foundation, a nonprofit co-founded by AI researcher François Chollet, revealed in a blog post on Monday that it has developed a new, highly challenging test to assess the general intelligence of advanced AI models.

    This new test, called ARC-AGI-2, has proven difficult for most models.

    According to the Arc Prize leaderboard, reasoning-based AI models such as OpenAI’s o1-pro and DeepSeek’s R1 scored between 1% and 1.3% on ARC-AGI-2. Meanwhile, high-performance non-reasoning models like GPT-4.5, Claude 3.7 Sonnet, and Gemini 2.0 Flash achieved around 1%.

    ARC-AGI Challenges AI with Unseen Pattern Recognition Tasks

    The ARC-AGI tests present puzzle-like challenges in which AI models must recognize visual patterns from grids of different-colored squares and generate the correct response grid. These tasks are specifically designed to test an AI’s ability to adapt to novel problems it has never encountered before.

    The Arc Prize Foundation had over 400 individuals attempt ARC-AGI-2 to establish a human baseline. On average, groups of participants correctly answered 60% of the test’s questions—significantly outperforming all AI models tested.

    a sample question from Arc-AGI-2 (credit: Arc Prize).

    In a post on X, François Chollet stated that ARC-AGI-2 provides a more accurate assessment of an AI model’s intelligence than its predecessor, ARC-AGI-1. The Arc Prize Foundation’s tests are designed to determine whether AI systems can effectively learn new skills beyond their training data.

    According to Chollet, ARC-AGI-2 eliminates the ability of AI models to rely on “brute force“—using vast computing power to solve problems—a significant weakness of ARC-AGI-1 that he previously acknowledged.

    To overcome the limitations of the first test, ARC-AGI-2 introduces a new key metric: efficiency. It also requires models to analyze patterns in real time rather than relying on memorization.

    “Intelligence isn’t just about solving problems or achieving high scores,” wrote Arc Prize Foundation co-founder Greg Kamradt in a blog post. “The efficiency with which these abilities are acquired and applied is a crucial factor. The real question isn’t just, ‘Can AI develop the skill to solve a task?’ but also, ‘At what cost and efficiency?’”

    Breakthrough and Limitations: OpenAI’s o3 Model Surpasses ARC-AGI-1 but at a High Cost

    For nearly five years, ARC-AGI-1 remained unbeaten until December 2024, when OpenAI’s advanced reasoning model, o3, surpassed all other AI systems and matched human performance on the test. However, as previously noted, these improvements came at a high computational cost.

    The first version of OpenAI’s o3 model to break records on ARC-AGI-1—o3 (low), which scored 75.7%—performed significantly worse on ARC-AGI-2, achieving only 4% while using $200 worth of computing power per task.

    Comparison of Frontier AI model performance on ARC-AGI-1 and ARC-AGI-2 (credit: Arc Prize).

    The launch of ARC-AGI-2 comes at a time when many in the tech industry are advocating for fresh, unsaturated benchmarks to track AI advancements. Hugging Face co-founder Thomas Wolf recently told TechCrunch that the AI field lacks adequate tests to assess key aspects of artificial general intelligence, such as creativity.

    In addition to introducing the new benchmark, the Arc Prize Foundation announced the Arc Prize 2025 contest, which challenges developers to achieve 85% accuracy on the ARC-AGI-2 test while keeping computational costs at just $0.42 per task.

    Read the original article on: TechCrunch

    Read more: New AI Identifies Nearly 100% Of Cancer, Outperforming Doctors