Anthropic Leads Push to Let AI Directly Control Users’ Computers, Sparking Industry Debate

Anthropic’s latest demo shows Claude actually doing the work people used to do with a mouse and keyboard, not just talking about it. In the new “computer‑use” preview, the model can move the cursor, click, scroll and type inside a live desktop session, then hand control back to the user through the Claude app on a phone. The company frames the feature as a “remote‑control” mode that lets a user schedule tasks—“send me a morning digest of Slack and email messages” or “suggest some times for a meeting”—and watch Claude execute them in real time. According to [Nymag], the demo is smoother than the 2024 proof‑of‑concept, which was “slow, error‑prone, and prone to quickly losing track of what it was doing.” The newer version, however, still relies on the user granting Claude visual access to the screen rather than deeper integration with codebases or APIs.

The shift from a clunky desktop‑manipulation trick to a more polished agent reflects how “agentic” AI has evolved since Anthropic’s first foray. In 2024, the industry’s imagination was limited to chat‑only bots that could not browse the web or interact with operating systems. By contrast, the current rollout leans on the “Claude Code” paradigm, which sidesteps the need to hijack a user’s screen and instead asks for direct access to code repositories, developer tools, and web resources. As [Nymag] notes, this approach “broadly reset the terms of the AI and is in the process of upending software development as we know it.” The move mirrors OpenAI’s Atlas browser experiment, but Anthropic’s angle is to embed the agent in everyday productivity tools—spreadsheets, slide decks, and email—rather than limiting it to a browser sandbox.

Anthropic isn’t the only player chasing the “self‑driving computer” dream. The article points to a growing ecosystem of user‑built agents, such as OpenClaw, which lets individuals script personal‑assistant bots that can act across email, social media and e‑commerce accounts. These tools give power users the ability to command a virtual assistant via chat, then let the agent carry out the instructions on the user’s machine. Anthropic’s “Cowork” product is positioned as a counterpart for the office crowd, tying Claude to the files, data and workflows that dominate desk‑bound jobs. By integrating the new computer‑use capability with Cowork, Anthropic hopes to turn Claude into a true “power‑user” assistant that can, for example, assemble a slide deck from raw data or reconcile a spreadsheet without the user lifting a finger.

Industry observers see the debate as a litmus test for how much control users will cede to AI. The 2024 test was dismissed as a “compelling demo” meant to gather developer feedback, according to [Nymag]. Today’s iteration, however, arrives amid heightened scrutiny over privacy, security and the potential for malicious automation. The article underscores that while the technology is “more fluid and competent,” it still requires explicit permission to view the desktop and can be commandeered remotely from a phone. That dual‑handedness—empowering users while exposing a new attack surface—has sparked a chorus of caution from security analysts, who warn that any misstep could let an AI agent execute unintended actions on a corporate network.

The conversation is now less about whether AI can click a button and more about how it should do so. Anthropic’s roadmap suggests a future where agents are less “screen‑hacking” and more “code‑integrated,” but the transition will hinge on trust. As [Nymag] puts it, the company is “testing its new computer‑use feature, alongside a tool that lets you take charge of your agent remotely through the Claude app.” Whether that remote‑control model will win over skeptics—or open the door to a new class of AI‑driven productivity tools—remains the central question driving the industry debate.