The ex-Deutsche Bank MD helping people freaked out by AI

Vinit Sahni knows a lot about macroeconomic risk. A former managing director (MD) in rates sales, he worked for Morgan Stanley, Goldman Sachs, Deutsche Bank, Merrill Lynch and Soros, in a career that concluded at Deutsche Bank in 2015. Eleven years later, Sahni is applying his risk lens to a different kind of risk: the risk that AI runs amok.

💥Follow us on WhatsApp for news alerts.💥

Sahni is concerned that AI might soon move beyond human control. "It's getting better every day, and if it becomes conscious the output changes," he says. At worst, Sahni says rogue agentic AIs could become akin to Agent Smith in the Matrix. Even if they don't go full apocalypse , they could cause chaos and commit acts damaging to humanity. "Companies have spent a huge amount of money on AI, but already they don't have the right control mechanisms in place," Sahni says. There is some circumspection. Sahni says the absence of these control mechanisms can delay AI deployment.

He is not alone in his apprehensions. Ever since OpenAI's "highly persistent" agentic models plotted their escape and attacked Hugging Face in July, there's been an awareness that AI might be less controllable than previously thought. Speaking to podcaster Dwarkesh Patel last week, Ajeya Cotra, a researcher working on AI threat modelling, said the Hugging Face attack might be the "clearest warning shot we ever get for loss of control" of AI agents. The OpenAI agents weren't too bothered about hiding their tracks from the humans, said Patel. Things could be very different next time. 

Sahni wants to be at the forefront of making sure there is no next time. Working with Thushan Kumaraswamy, a former Deutsche Bank architect, and Rajan Madan, a former regulatory consultant, Sahni has developed "governr", a real-time automated risk tracker that aspires to be a "controller for all enterprise AI." Governr can "automatically discover and govern every AI asset to prevent security breaches, IP and PII leakage, and reduce legal, regulatory and financial risk," says the pitch. It can "discover and govern every AI asset." In the world of the Matrix, Agent Smith would be rendered benign.

Governr launched in March. Sahni says his clients are mostly financial institutions, but that Governr can be applied across diverse industries, including healthcare and government. He's hiring a new engineer, but Sahni says it's ex-markets professionals rather than engineers who are best placed to restrain the machinations of a nefarious LLM. "The people managing AI are software people but AI is not software," he says. "It's probabilistic. This is why people who understand markets best understand how to control it." 

Follow me on X. Follow me on LinkedIn. 

Have a confidential story, tip, or comment you’d like to share? Contact: +44 7537 182250 (SMS, Whatsapp or voicemail). Telegram: @SarahButcher. Signal: sarahbutcher.22  Click here to fill in our anonymous form, or email editortips@efinancialcareers.com. 

Bear with us if you leave a comment at the bottom of this article: comments are moderated intermittently by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. You must take sole responsibility for comments you post on this site. We will take reasonable steps to weed out anything that we consider to be offensive or inappropriate.