The Silicon Valley has lately been embroiled in a lobbying battle between big tech firms including Meta, Mistral and IBM on one side advocating for an “open science” approach to AI development that puts them at odds with rivals Anthropic, Microsoft and ChatGPT-maker OpenAI who are backing closed software. In fact, OpenAI is often trolled on social media for not being true to its name.

But as India starts to emerge as the use-case capital of GenAI applications, tech leaders and startup founders believe both technologies have a unique role to play in optimising costs, ensuring data sovereignty and delivering the best performance.

ET spoke to a cross section of startups and enterprises to understand the benefits and challenges of using open versus closed-source GenAI models.

Cost of Deployment

By definition, open-source software is where the source code is available to everyone in the public domain to use, modify, and distribute. Closed source, on the other hand, means the source code is restricted to private use and cannot be altered or built upon by users.

In the context of GenAI, while open models are free and flexible, their cost of deployment may outrun the closed ones, industry experts say.

“If you want to host an open model on premise, fine-tune it and customize it for organization-specific needs, there is a substantial infrastructure cost, LLMOps, lifecycle management and inferencing cost involved,” said Arun Chandrasekaran, Gartner Distinguished VP Analyst.

“Open-source models, while often free to access, can incur high deployment and maintenance costs due to technical requirements,” said Rashid Khan, co-founder, of conversational AI startup Yellow.ai.

Conversely, closed API models are often optimized, well-maintained, and continuously updated, which can save time and resources. “They provide a more turnkey solution with dedicated vendor support, ease of integration, and regular updates,” he said.

In fact, over the last 12 months, the pricing of closed LLM APIs has come down by 65-90%, which is helping startups to expand their margins, invest in research and development, improve performance, hire more talent as well as price their products competitively.

“Deploying and fine-tuning an open-source model is a cost to the company and for initial stages of low volume or with some use cases that do not require 100% availability of the LLM, preferred approach is to go with paid APIs,” said Ankur Dhawan, President – Product and Technology, edtech company upGrad.

However, if an organization has the expertise and resources to manage the infrastructure optimally and technical skills to fine tune, they can eliminate the recurring costs associated with API access of closed models.

For instance, Gnani.ai, which develops customer service chatbots and voice bots says the tokens consumed in a closed model often determine the costs. “In contrast, if you have a fine-tuned open model with your own proprietary data, you may have less to worry about regarding costs except for the deployment,” said Ganesh Gopalan, Co-Founder and CEO, Gnani. ai.

Performance

AI researchers and scientists are divided over performance benchmarking. Stanford studies showed closed-source models still outperform their open-sourced counterparts. But common consensus suggests open models are catching up fast. The best closed models are getting better; and the best open models are getting better faster than them.

“We had previously seen certain closed source models such as GPT-4o leading the charge in terms of quality, but open-source models are quickly catching up. For example, Meta’s recently announced Llama 3.1 405B model is the same quality as other top models like GPT-4o and Claude Sonnet 3.5,” said Baris Gultekin, Head of AI, Snowflake, US-based data cloud company which hosts top GenAI models.

Meta, which is leading the open-source revolution with its Llama models, believes organizations must be more transparent about their evaluations.

“We at Meta not only publish the benchmarks, but also the methodology that goes above and beyond what a lot of the other proprietary vendors do,” said Ragavan Srinivasan, Vice President – Product Management, Meta. “…people aren’t as open and transparent. There’s just a scorecard. How are we evaluating this? Did you use 10 prompts? What are those 10 prompts? Are they all similar?”

When it comes to critical applications, especially with agentic AI, controlling hallucinations and inaccuracies is the key priority for enterprises to preserve brand value.

“It’s about the trade-off between cost and quality,” says Jonathan Frankle, Chief AI Scientist at Databricks, one among the highest valued AI companies. He added that cost takes many different forms. This could be speed or latency. “For coding assistants and real time chatbots, speed matters a lot.”

“What we’ve seen in the past few months is that this tradeoff has gotten better for everyone. For any given amount of money that you spend, you’re going to get a better model,” he said.

Data sovereignty and privacy

“When I speak with organizations, they often prefer open-source models,” Snowflake’s Gultekin said. “Companies want to bring AI closer to their data, ensuring that data security and privacy are upheld. Where the model runs is really important.”

For instance, banks are not comfortable with their data being exchanged to a public cloud where these closed models are accessible, says Gartner’s Chandrasekaran.

“But tomorrow if the same sits on-premise through some licensing arrangement, it could solve data security or privacy concerns,” he added.

Gnani.ai, which works with BFSI and healthcare organizations, said, for mission-critical applications, customers deploy on private clouds or even on-premise servers for customers in regulated industries in both US and India, Gopalan said.

Open models are winning here. There’s an intrinsic limitation to the amount of transparency one can provide with closed models, Databricks’ Frankle said.

“So, for me, as Databricks, I love the idea that I can sit down with a customer and walk them through what happens in our systems and how everything operates, and the model weights are available to them.”

“LLMs are black boxes, you can’t exactly figure out why and how it does what it does,” UpGrad’s Dhawan said. “Hence, the decision on which one to use will depend on other factors like maintenance, availability, and cost of running the model. The trend is going to be similar to utilising the cloud for storing data as opposed to on-prem.”

Orchestration layer

There is no one-size-fits-all. Therefore, organizations are experimenting with a multi-vendor strategy which orchestrates through cost, performance and security to offer the required outcomes.

“A lot of CIOs or organizations are today looking at a multi-model strategy because they want to reduce vendor lock-in to any single provider. But the critical question is how I align the right model for right use-cases,” Gartner’s Chandrasekaran said.

“So if you have a model garden, you need an orchestration tool in the backend, which not only optimizes my cost but also gives the best output through automated selection,” he said.

Microsoft Azure, the leading cloud provider which today hosts more than 1600 open and closed models, says “…factors like cost, efficiency, latency, and accuracy, make choosing the right model a critical step for enterprise adoption.”

“That’s why Microsoft is committed to giving our customers flexibility in how they can develop and deploy customized AI solutions – either off the shelf or fine-tuned – through Azure AI,” a company spokesperson said.

Google, which has led the open-software revolution with Android OS powering millions of mobile devices, today, stands at a crucial intersection of open and closed systems when it comes to GenAI. It has a family of both open (Gemma) and closed (Gemini) models. In fact, back in 2019, Google scientists co-authored the seminal paper introducing the transformer architecture which today forms the basis of all GenAI models. It has also built key developer tools and infrastructure products like Transformer, TensorFlow and Jax needed for engineering these models.

“Contributing openly to the research ecosystem is deeply embedded in our DNA…Decisions around whether to open source or release models to the public need to be made on a case-by-case basis,” a Google spokesperson said. “It’s important to carefully assess the balance of potential benefits and harms before releasing any AI systems to the general public.”