Generative AI has been around since the 1960s in the form of chatbots. However, it was the introduction of generative adversarial networks or GANS in 2014 that made it possible for generative AI to create convincingly authentic images, videos and audio.
Fast forward to 2024 and attention is now focused on foundation models and the role these play as organisations race to have in place a successful AI strategy.
Gartner defines foundation models as large-parameter models that are trained on a broad gamut of datasets in a self-supervised manner.
“They are called foundation models because of their critical importance and applicability to a wide variety of downstream use cases. This broad applicability is due to the pretraining and versatility of the models.”
Gartner
Luv Aggarwal, worldwide sales leader for IBM watsonx.ai at IBM, suggests thinking of the process as collecting data from multiple sources, including the internet and filtering it down to a model that can understand general language. He clarifies that large language models (LLMs) are a type of foundation model.
Adoption trends
Aggarwal acknowledged that enterprises have been trying to adopt AI into various business processes. “However, in the last couple of years, with the adoption of these very powerful foundation models, enterprises are accelerating (foundation model) deployments, including use cases such as customer care, HR, and marketing,” he elaborated.
Aggarwal expressed excitement over the development of LLMs specialised in Southeast Asian languages. “They are taking base models from (for example LLhama from Meta) and adapting to local and regional languages. We can expect to see use cases developed in the US to be localised and adopted in the Asian markets at a faster pace,” he opined.
Aggarwal clarifies that local languages have nuances, for example, some terminologies or statements contain English together with the local language or languages. “You need foundation models that are trained on certain dialects and can adapt to mixes of English, local languages and colloquial terms used in that country that don’t translate well into English,” he explained.
Goals and overcoming hurdles
While the allure of deploying foundation models exists, there remain multiple obstacles in the way of success. From data scarcity, machine learning bias, scaling issues, and certainly the limited availability of skills and experience in the technology and the workflow.
Aggarwal says the goals of foundation models have not changed from the early days of artificial intelligence – to bring to the surface actionable insights more quickly. “The challenges today are the same as in the past. How do you build a robust data foundation to power those kinds of use cases,” he begins.
“If you don't know where your data is (coming from), if you don't know the quality of your data if you don't know how to pipe and integrate it to get it to the right place, you are not going to get valid outputs from generative AI workloads. It is garbage in, garbage out.”
Luv Aggarwal
Maturing AI use creates new challenges
Aggarwal observed that with each new technology, there is a ‘quick ramp-up’ alongside the noise and excitement that comes with the innovation. He posits that as enterprises mature in their understanding and experience with technologies including adopting business-critical use cases, new challenges such as operationalising the technology reveal themselves.
“With every new release of a more complex large language model, we are hitting that point of diminishing return where the expectations are so high now for these new models and as they come out the performance enhancements and improvements are less so compared to the previous model,” he elaborates.
To complicate matters, he posits that enterprises are the limit in terms of compute resources and available data to train these models.
“Yes, very large language models with a lot of parameters are great but enterprises are reaching a limit in terms of how much compute they can have access to. So, enterprises, using their proprietary data, will need to be content with smaller, more targeted models and more efficient,” argues Aggarwal.
Click on the video to watch the full discourse with Aggarwal.
- What is the relationship between foundation models and generative AI?
- How have organisations leveraged foundation models and generative AI to enhance various aspects of their operations? AI adoption trends in the US versus in ASEAN.
- What are the key successes organisations hope to achieve using generative AI? What stops them from achieving their goals?
- Generative AI has already reached its “hobbyist” phase—and as with computers, further progress aims to attain greater performance in smaller packages.
- There is no one-size-fits-all, organisations want the right models for the right use cases. They must also be able to use their model based on their needs.
- Examples of how the use of customized language models can bring better benefits that generic models may miss.
- There has been a lot of discussion recently about the potential risks of AI. Will the use of BYOM bring more risk to organisations?
