Technology and ethics Recent blog posts

Technology and ethics

Robots taking the decision to kill


 

http://www.nytimes.com/2014/11/12/science/weapons-directed-by-robots-not-humans-raise-ethical-questions.html?_r=0

 

Fearing Bombs That Can Pick Whom to Kill

By JOHN MARKOFFNOV. 11, 2014

Inside

Photo

A Long Range Anti-Ship Missile prototype, launched by a B-1 bomber, is designed to maneuver without human control. Credit Defense Advanced Research Projects Agency

Continue reading the main story Share This Page

Continue reading the main story

Continue reading the main story

Continue reading the main story

Continue reading the main story

On a bright fall day last year off the coast of Southern California, an Air Force B-1 bomber launched an experimental missile that may herald the future of warfare.

Initially, pilots aboard the plane directed the missile, but halfway to its destination, it severed communication with its operators. Alone, without human oversight, the missile decided which of three ships to attack, dropping to just above the sea surface and striking a 260-foot unmanned freighter.

Warfare is increasingly guided by software. Today, armed drones can be operated by remote pilots peering into video screens thousands of miles from the battlefield. But now, some scientists say, arms makers have crossed into troubling territory: They are developing weapons that rely on artificial intelligence, not human instruction, to decide what to target and whom to kill.

As these weapons become smarter and nimbler, critics fear they will become increasingly difficult for humans to control — or to defend against. And while pinpoint accuracy could save civilian lives, critics fear weapons without human oversight could make war more likely, as easy as flipping a switch.

Continue reading the main story Video

Play Video|1:29

Animation of How New Missiles May Work

Animation of How New Missiles May Work

In an artistic rendering from Lockheed Martin, tactical missiles that operate using artificial intelligence are shown selecting and destroying their targets.

Publish Date November 11, 2014. Photo by Lockheed Martin.

Britain, Israel and Norway are already deploying missiles and drones that carry out attacks against enemy radar, tanks or ships without direct human control. After launch, so-called autonomous weapons rely on artificial intelligence and sensors to select targets and to initiate an attack.

Britain’s “fire and forget” Brimstone missiles, for example, can distinguish among tanks and cars and buses without human assistance, and can hunt targets in a predesignated region without oversight. The Brimstones also communicate with one another, sharing their targets.

Armaments with even more advanced self-governance are on the drawing board, although the details usually are kept secret. “An autonomous weapons arms race is already taking place,” said Steve Omohundro, a physicist and artificial intelligence specialist at Self-Aware Systems, a research center in Palo Alto, Calif. “They can respond faster, more efficiently and less predictably.”

Continue reading the main story

Concerned by the prospect of a robotics arms race, representatives from dozens of nations will meet on Thursday in Geneva to consider whether development of these weapons should be restricted by the Convention on Certain Conventional Weapons. Christof Heyns, the United Nations special rapporteur on extrajudicial, summary or arbitrary executions, last year called for a moratorium on the development of these weapons.

The Pentagon has issued a directive requiring high-level authorization for the development of weapons capable of killing without human oversight. But fast-moving technology has already made the directive obsolete, some scientists say.

“Our concern is with how the targets are determined, and more importantly, who determines them,” said Peter Asaro, a co-founder and vice chairman of the International Committee for Robot Arms Control, a group of scientists that advocates restrictions on the use of military robots. “Are these human-designated targets? Or are these systems automatically deciding what is a target?”

Continue reading the main story

Weapons manufacturers in the United States were the first to develop advanced autonomous weapons. An early version of the Tomahawk cruise missile had the ability to hunt for Soviet ships over the horizon without direct human control. It was withdrawn in the early 1990s after a nuclear arms treaty with Russia.

Back in 1988, the Navy test-fired a Harpoon antiship missile that employed an early form of self-guidance. The missile mistook an Indian freighter that had strayed onto the test range for its target. The Harpoon, which did not have a warhead, hit the bridge of the freighter, killing a crew member.

Despite the accident, the Harpoon became a mainstay of naval armaments and remains in wide use.

In recent years, artificial intelligence has begun to supplant human decision-making in a variety of fields, such as high-speed stock trading and medical diagnostics, and even in self-driving cars. But technological advances in three particular areas have made self-governing weapons a real possibility.

New types of radar, laser and infrared sensors are helping missiles and drones better calculate their position and orientation. “Machine vision,” resembling that of humans, identifies patterns in images and helps weapons distinguish important targets. This nuanced sensory information can be quickly interpreted by sophisticated artificial intelligence systems, enabling a missile or drone to carry out its own analysis in flight. And computer hardware hosting it all has become relatively inexpensive — and expendable.

The missile tested off the coast of California, the Long Range Anti-Ship Missile, is under development by Lockheed Martin for the Air Force and Navy. It is intended to fly for hundreds of miles, maneuvering on its own to avoid radar, and out of radio contact with human controllers.

Photo

Images from a computer showing a strike by a Brimstone missile, a British weapon, on an Islamic State armed truck in Iraq. The “fire and forget” missile can distinguish among tanks and cars and buses without human assistance. Credit Ministry of Defense/Crown Copyright, via Associated Press

In a directive published in 2012, the Pentagon drew a line between semiautonomous weapons, whose targets are chosen by a human operator, and fully autonomous weapons that can hunt and engage targets without intervention.

Weapons of the future, the directive said, must be “designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force.”

The Pentagon nonetheless argues that the new antiship missile is only semiautonomous and that humans are sufficiently represented in its targeting and killing decisions. But officials at the Defense Advanced Research Projects Agency, which initially developed the missile, and Lockheed declined to comment on how the weapon decides on targets, saying the information is classified.

Continue reading the main story

Recent Comments

Paul Wedel

4 hours ago

The development of AI weapons is one aspect of progress towards highly intelligent AI systems. Such systems could potentially become able to...

cfranck

7 hours ago

The critics for decades deplored the inhumanity of collateral damage of non-precision weapons. Then came precision weapons, and deplored...

vulcanalex

7 hours ago

Seems great to me, we get the job done better and with less risk. Machines can take G forces better than humans and they can respond much...

  • See All Comments

“It will be operating autonomously when it searches for the enemy fleet,” said Mark A. Gubrud, a physicist and a member of the International Committee for Robot Arms Control, and an early critic of so-called smart weapons. “This is pretty sophisticated stuff that I would call artificial intelligence outside human control.”

Paul Scharre, a weapons specialist now at the Center for a New American Security who led the working group that wrote the Pentagon directive, said, “It’s valid to ask if this crosses the line.”

Continue reading the main story

Continue reading the main story

Some arms-control specialists say that requiring only “appropriate” human control of these weapons is too vague, speeding the development of new targeting systems that automate killing.

Mr. Heyns, of the United Nations, said that nations with advanced weapons should agree to limit their weapons systems to those with “meaningful” human control over the selection and attack of targets. “It must be similar to the role a commander has over his troops,” Mr. Heyns said.

Systems that permit humans to override the computer’s decisions may not meet that criterion, he added. Weapons that make their own decisions move so quickly that human overseers soon may not be able to keep up. Yet many of them are explicitly designed to permit human operators to step away from controls. Israel’s antiradar missile, the Harpy, loiters in the sky until an enemy radar is turned on. It then attacks and destroys the radar installation on its own.

Norway plans to equip its fleet of advanced jet fighters with the Joint Strike Missile, which can hunt, recognize and detect a target without human intervention. Opponents have called it a “killer robot.”

Military analysts like Mr. Scharre argue that automated weapons like these should be embraced because they may result in fewer mass killings and civilian casualties. Autonomous weapons, they say, do not commit war crimes.

Continue reading the main story Write A Comment

On Sept. 16, 2011, for example, British warplanes fired two dozen Brimstone missiles at a group of Libyan tanks that were shelling civilians. Eight or more of the tanks were destroyed simultaneously, according to a military spokesman, saving the lives of many civilians.

It would have been difficult for human operators to coordinate the swarm of missiles with similar precision.

“Better, smarter weapons are good if they reduce civilian casualties or indiscriminate killing,” Mr. Scharre said.

A version of this article appears in print on November 12, 2014, on page A1 of the New York edition with the headline: Fearing Bombs That Can Pick Whom to Kill. Order Reprints|Today's Paper|Subscribe

Close this panel « Expand this panel » Collapse this panel

265 Comments

Readers shared their thoughts on this article.

The comments section is closed. To send a letter to the editor, write to letters@nytimes.com.

  • All  265
  • Readers’ Picks  136
  • NYT Picks  15

 

 
Posted by Alon Serper on 13 November 2014 10:31:43


Current rating: 0 (0 ratings)

Bookmark this page to:Add to Live Add to Google Bookmarks Add to Delicious Add to Digg Add to Blogmarks Add to StumbleUpon Add to Yahoo Bookmarks Add to Terchnorati Add to Twitter Add to Yahoo MyWeb Add to Reddit Add to Facebook

Comments
Blog post currently doesn't have any comments.

Sign in


By signing in you accept these
Terms of Use

Latest blog comments

15/04/2018
Nic Wickens
THoughtful and interesting

14/07/2014
Alon Serper
Let us understand the other side, engage him or her, put ou...

14/07/2014
Alon Serper
Thank you Sameera, Racism is inexcusable. There canno...

14/07/2014
Sameera Patel
It is always heartening to come across a non-mainstream vie...

12/06/2013
Online Community Manager
These confessions pages are certainly a very interesting (i...

 

Telephone: 041-504 1111 | Fax: 041-504 2574 / 2731 | E-mail: info@mandela.ac.za

PO Box 77000 | Nelson Mandela University | Port Elizabeth | 6031 | South Africa