Times columnist Kevin Roose sees reasons for optimism — and worry — as the world wakes up to what Silicon Valley has been talking about for years.
The New York Times•Kevin Roose18 Sept
In recent weeks, leaders of some of the biggest AI companies have warned that the very tech they are developing is dangerous. Last weekend, Anthropic CEO Dario Amodei argued that AI carries serious risk and that progress should be slowed. OpenAI CEO Sam Altman responded on X: “I agree with Dario that we need to pace the…
MIT Technology Review•Jessica Hamzelou18 SeptGeoffrey Hinton's warning came days after Anthropic CEO Dario Amodei published an essay urging the AI industry to "pace the frontier."
Anthropic's Claude is helping the company develop the next, more intelligent version of the model, the artificial intelligence lab said in an announcement Thursday. Claude is leading 26% of Anthropic's model research and development, which the company said means it can complete most of a given task "end-to-end from a high-level prompt" while still being under human supervision. The model is not yet working completely autonomously. Still, about 90% of the company's research and development is done in "collaboration” with Claude, which Anthropic said means the model can do "large chunks of work under close human direction.” The announcement came as some leading figures in AI, led in large part by Anthropic CEO Dario Amodei, are calling for a slowdown in the technology’s development over safety concerns. As leaders consider pacing AI's development, "we should do everything possible to minimize the gap between what frontier labs know and what the public knows,” the company said in a blog post. "This means better measuring the development of AI, reporting on it publicly, and giving society an opportunity to decide how to use this information.” The company said that models accelerating their own development could make it "more challenging for humans to understand or control these systems." It argued that sharing these metrics could lead to better understanding of how close leading AI labs are to reaching recursive self-improvement, or a model's ability to autonomously build its successor. Anthropic also urged other AI developers to share similar metrics on a regular basis, encouraging the use of a public methodology so the numbers could be compared over time, and potentially across labs. It was unclear from Anthropic's disclosure how close the company believes it is to achieving recursive self-improvement, but the pace at which Claude has increasingly contributed to research and development is notable. The portion of work Claude "leads,” or does in large part while remaining under human supervision, was none in February. Six months later, it was leading a quarter of research and development work, reaching that benchmark in August. The company also shared details of agent oversight measures it has in place, noting that there were approximately 30,000 agents doing research and engineering work as of August. Oversight measures are crucial for seeing how often agent misbehavior is detected by monitoring systems, the company said. Anthropic recently committed to setting up external third-party evaluators who will be embedded within the company to monitor safety efforts. An Anthropic researcher kicked off much of the recent dialogue around AI safety when he resigned last week with a dire warning about the threats the technology poses to humanity. Amodei, OpenAI CEO Sam Altman, Elon Musk and other tech leaders have since supported the idea of slowing down development, but other tech leaders and President Donald Trump have pushed back.
Not everyone agrees with Amodei's call for globally coordinated action for AI safety.
TechCrunch•Lucas Ropek17 Sept
Skepticism over antitrust waivers highlights the tension between fostering AI safety collaboration and maintaining competitive market dynamics.
CryptoBriefing•Editorial Team17 Sept
Anthropic CEO Dario Amodei’s call for an antitrust waiver for AI firms to work together on a safety standard is being met with skepticism in Washington, where policymakers are questioning tech leaders’ intentions. Congress has failed repeatedly to pass AI regulations for years, but some lawmakers are balking at the idea of technology companies taking…
The Hill•Miranda Nazzaro17 Sept
The collected essays of Dario Amodei help explain why some people are so worried about artificial intelligence.
The New York Times•David Streitfeld17 Sept
Anthropic CEO Dario Amodei's call to slow AI development won quick backing from OpenAI, xAI, and DeepMind, but critics see competitive motives
CryptoBriefing•Editorial Team16 Sept
On Saturday, Anthropic CEO Dario Amodei published an essay urging tech leaders to slow the pace of AI development. On a Sunday CBS News interview, he added that it “could be a good idea” to have an AI kill switch — not unlike what Rep. Ted Lieu (D-Calif.) and Rep. Nathaniel Moran (R-Texas) have proposed. “Some…
The Hill•Kim Wehle, opinion contributor16 Sept
NYC's AI safety hearing could set a precedent for local tech regulation, influencing broader legislative trends on AI oversight and accountability.
CryptoBriefing•Editorial Team16 Sept
Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it's time to make everyone slow down before we lose control - including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X […]
The Verge•Sean Hollister16 Sept
Tech CEOs banding together is an old ruse recycled from corporate America to get a pass from antitrust laws Anthropic’s Dario Amodei is not the first corporate CEO to suggest that excessive competition is driving the world to some socially undesirable outcome. The safety breach disclosed by OpenAI after a swarm of its agents coordinated to breach their supposedly secure sandbox, get on the Internet and hack AI platform Hugging Face, warrants urgent action. It demonstrated the ease with which the technology can evade human control and gave concrete form to the existential fears about what it could do to humanity if not securely leashed. Continue reading...
The Guardian•Eduardo Porter16 Sept
US vice-president’s comments come as former Anthropic researcher revisits recent claim AI could destroy humanity The US vice-president has dismissed calls for global regulation of AI safety risks, telling companies creating the most advanced models: “If you’re building Frankenstein, stop.” In remarks addressed towards Dario Amodei, the co-founder of Anthropic who has called on Washington DC to coordinate control of AI systems , including with China, JD Vance said: “If you’re gonna create Frankenstein, don’t come to the government and say we need regulation.” Continue reading...
The Guardian•Robert Booth UK technology editor16 Sept
Trump and Republicans want companies to regulate themselves. It’s a dereliction of duty that will make AI less safe It is hard to get Sam Altman, Elon Musk and Dario Amodei to agree on much. But over the weekend, all three AI company CEOs called for AI development to slow down in the face of growing, alarming risks. Their employees are sounding the siren too, with one researcher publicly quitting and accusing OpenAI and Anthropic of “gambling with our lives”. The combination of dire warnings from insiders and growing real-world evidence of rogue AIs should, in a sane world, lead to government action. Instead, Donald Trump and the Republican leadership have their heads in the sand. Continue reading...
The Guardian•Shakeel Hashim16 Sept
The stark reality is this: we don’t know how to maintain control of these systems. But there are actions we can take With Jacob Coxon’s resignation from Anthropic , we have reached the AI risk tipping point. Millions of people are finally coming to understand what experts have known for years: AI companies have been gambling with all of our lives, and the odds are not good. In response, the Anthropic CEO, Dario Amodei, has introduced a proposal for “pacing the frontier”, endorsed by Sam Altman and Elon Musk. Can we breathe a sigh of relief? Are we about to step back from the edge of extinction? Amodei envisions a slowdown of one to two years, resulting in “profound progress” on technical safety measures. But that’s not what humanity needs right now. What we need is a plan in which we’re confident AI isn’t going to kill us, and this isn’t it. Continue reading...
The Guardian•David Krueger16 Sept
A product conference at software giant Salesforce turned into the battleground on the new debate on whether there should be a slowdown in artificial intelligence.
Long-time bitcoin bull Jack Dorsey just pushed back on parts of Anthropic’s boss’s AI development pacing plan, advocating for more openness. How this debate settles could also affect Dorsey-led Block, whose products span bitcoin and AI-powered solutions as the company builds a financial system “open to everyone.” Claude developer Anthropic’s CEO Dario Amodei recently started […]
Bitcoin.com News•Linas Kmieliauskas16 Sept
In today’s newsletter: It is hard to tell fact from fiction when it comes to AI. What is really going on – and what should the government do about it? Good morning. As a general rule, it pays to be suspicious of any gigantic company that claims it’s developing a tool capable of destroying humanity. But in recent days, a number of warnings from the AI industry have suggested that even tech insiders are starting to worry about what they have unleashed. In a lofty essay published on Saturday, Dario Amodei, the founder of Anthropic (the company behind Claude), argued that tech companies need to “slow” the pace at which they’re developing the newest and most sophisticated AI models. Lucy Letby | Three babies might have survived if hospital had acted upon concerns over Lucy Letby, an inquiry has found . Lady Justice Thirlwall condemned the ‘complete failure’ to protect babies on neonatal unit at Countess of Chester hospital. UK politics | The leader of Reform UK in Wales stood down after arrest on suspicion of assault . Dan Thomas, a former Tory councillor, was elected to Senedd as leader of the opposition in May. AI | The progressive senator Bernie Sanders and rightwing strategist Steve Bannon have called for restrictions on artificial intelligence (AI) but offered competing visions for what they termed a “cold war” with China. UK news | Young people in Rotherham face a local jobs market with the fewest suitable opportunities in Britain, according to a report that warns stark regional divisions are fuelling a crisis in youth work. Defence | John Healey is in talks with the Canadian government about joining a new global defence bank intended to help allies rearm to counter mounting security threats, just weeks after Rachel Reeves rejected the move. Continue reading...
The Guardian•Hettie O'Brien16 Sept
Meta CEO Mark Zuckerberg's view of artificial intelligence safety is akin to the perspective of Nvidia's Jensen Huang rather than Anthropic chief Dario Amodei.