The AI tools that crunch numbers, generate text and videos and find patterns in data rely on mass surveillance and exercise concerning control over our lives, the boss of encrypted messaging app Signal told AFP on Thursday.

Pushing back against the unquestioning enthusiasm at VivaTech in Paris, Europe’s top startup conference where industry players vaunt the merits of their products, Meredith Whittaker said concerns about surveillance and those about AI were “two framings of the same thing”.

“The AI technologies we’re talking about today are reliant on mass surveillance,” she said.

“They require huge amounts of data that are the derivatives of this mass surveillance business model that grew out of the 90s in the US, and has become the economic engine of the tech industry.”

Whittaker, who spent years working for Google before helping to organize a staff walkout in 2018 over working conditions, established the AI Now Institute at New York University in 2017.

She now campaigns for privacy and rails against the business models built on the extraction of personal data.

And she is clear that she has no confidence that the AI industry is developing in the right direction.

Power imbalances

AI systems have a hunger for data to input but they produce vast amounts of data too.

Even if it is incorrect, she said, this output “has power to classify, order and direct our lives in ways that we should be equally concerned about”.

And she pointed to the power imbalances created by an industry controlled by “a handful of surveillance giants” that are “largely unaccountable”.

“Most of us are not the users of AI,” she said.

“Most of us are subjected to its use by our employers, by law enforcement, by governments, by whoever it is.

“They have their own goals but they may not be goals that benefit us or benefit society.”

She said a striking example was the way AI firms liked to say that they were helping to find solutions to the climate crisis.

In fact, she said, they were taking money from fossil fuel companies and their technology was being used to find new resources to extract.

“Because, of course, where is the revenue? It’s not in saving the climate,” she said.

“It is in massive contracts with BP, with Exxon, with other large oil and gas companies.”

Ultimately she argued that Europeans should not be thinking in terms of competing with bigger American AI firms.

Another option could be “to reimagine tech that can serve more democratic and more rights-preserving or pluralistic societies”.