No technology is perfect. Even self-driving cars trained to obey traffic laws are bound to run into issues that cause them to commit a citable offense. Such was the case with a Cruise-operated hatchback in San Francisco last weekend, which was pulled over by local law enforcement for failing to switch on its headlights. While the car came to a stop, as video of the incident shows, there’s policy to be established when it comes to interactions between autonomous vehicles and police.

Originally published on Instagram, the video shows the car — one of Cruise’s Chevy Cruises — in the city’s Richmond District pulling over to the side of the road when signaled to do so by an officer, ahead of an intersection. The policeperson walks toward the car and attempts unsuccessfully to open the driver-side door, at which point the Cruise vehicle begins to drive down the road — only to pull over again and activate its hazards. Police approach the car a second time in a presumed effort to figure out how to turn on the headlights.

Cruise, which a little over two months ago began letting San Francisco residents hail rides in its driverless vehicles, asserts that the pulled-over vehicle acted as intended. The headlights indeed malfunctioned — and have been fixed, according to the company — but the car yielded to police and then pulled over to the “nearest safe location.” One of the officers contacted Cruise after the traffic stop, and no citation was issued.

But the episode raises questions about procedure where self-driving vehicles are involved with the police.

As it turns out, the vehicle in the video wasn’t entirely autonomous. Cruise in 2019 began testing computer vision and sound detection AI to help its cars respond to emergency vehicles. However, a spokesperson told TechCrunch that Cruise personnel directed the car to pull over at the second location — across the intersection — when it became clear that the car was the subject of a traffic stop and the officer was clear of the car. When asked, the spokesperson declined to reveal whether Cruise vehicles would behave differently if the stop happened on a highway versus a city street, for example, saying only that the company has a team of people to feed Cruise vehicles with information in instances where the vehicles experience problems.

The most common contact people in the U.S. have with police takes the form of a traffic stop. But not all drivers are treated equally. Black drivers are almost twice as likely to be pulled over as white drivers, far too often with deadly consequences. Several experts argue that autonomous cars will de-escalate interactions with the police by, for instance, reducing investigations associated with hit-and-run offenses. But others, including Elizabeth E. Joh, a law professor at UC Davis, expect that autonomous vehicles will increase police powers if police are allowed to seize cars remotely.

“An autonomous car would be a programmable car. Perhaps speed enforcement will be a thing of the past, either because cars will be programmed to stay within the legal speed limit, or because such violations will be automatically enforced with a ticket sent to drivers’ electronic dashboards,” Joh writes in an essay on the subject. “But even in this version of the future, police will still seize cars. A person inside the autonomous car may have an outstanding arrest warrant. The police may suspect that the car contains contraband or evidence of a crime. By no longer requiring human control, autonomous cars may even encourage more types of crime to take place within them.”

While no company developing autonomous vehicles has indicated that they’d provide law enforcement this sort of access, police might make the case for it, citing the confusion that can arise from incidents like the Cruise traffic stop. Cruise’s isn’t the first autonomous car to get pulled over — that distinction goes to a Google prototype in 2015. But as autonomous vehicle developers increasingly deploy their fleets on public roads, the issue is more likely to come to the fore.

“The police and the public will interact differently when there is little human involvement in driving,” Joh continues in the essay. “[B]ecause so much of the Fourth Amendment is premised upon human drivers and human police officers, that means that a future of automated car stops will pose novel and difficult questions of law and policy that we should begin to address now.”