Recent market trends have spotlighted the practical efficacy of generative artificial intelligence initiatives within corporate environments. A report from NANDA, an artificial intelligence company, has revealed that a significant majority of generative AI pilot programs do not translate into tangible financial benefits for businesses. This revelation has prompted a re-evaluation of expectations surrounding the economic contributions of advanced AI solutions.
On the trading floor, technology company shares experienced a notable downturn. The NASDAQ Composite index registered a 1.4% decrease, with prominent AI-focused entities such as Palantir and Arm Holdings seeing their values decline by 9.4% and 5% respectively. This market movement coincided with the release of the NANDA report, which detailed the high rate of projects failing to achieve substantial financial returns, marking the most significant single-day market drop since early August.
NANDA, originally an offshoot of the Massachusetts Institute of Technology Media Lab, describes its mission as developing an 'agentic web.' Their research paper, although now accessible primarily behind a paywall, indicates that a mere 5% of generative AI pilot projects successfully transition to full production and yield discernible monetary advantages. The comprehensive study was built upon 52 in-depth interviews with corporate decision-makers, an examination of over 300 public AI endeavors, and a questionnaire completed by 153 business leaders. The research rigorously tracked the return on investment six months following the conclusion of the pilot phases for these AI projects.
Interestingly, the study pointed out that while many organizations deploy AI in client-facing roles, the most financially beneficial applications often emerge from internal, back-office operations. It is in streamlining routine administrative functions where AI demonstrates its capacity to generate savings, predominantly by reducing reliance on external service providers. The survey, however, found no significant impact of AI projects on the overall headcount of internal personnel.
Despite a high proportion of employees (90%) acknowledging personal advantages from using publicly available AI models, such as large language models (LLMs) like ChatGPT, these individual gains rarely translate into institutional-level benefits. Approximately 40% of the companies surveyed currently invest in subscriptions to LLMs, indicating widespread adoption even without clear organizational financial upside.
A critical factor contributing to the failure of many generative AI projects, according to the surveyed organizations, is the models' inability to maintain contextual awareness. This deficiency manifests as a struggle to adapt to evolving circumstances, learn from prior interactions, or recall previous queries. NANDA’s findings suggest that forging strategic alliances with partners capable of providing adaptive, contextually intelligent AI systems is paramount for achieving successful implementation. Interviewees’ sentiments, with 60%-70% concurrence, highlighted issues such as AI systems not incorporating feedback and requiring excessive manual context for each interaction.
Sector-wise, the media and telecommunications industry showed the most positive impact from generative AI, followed by professional services, healthcare and pharmaceuticals, consumer and retail, and financial services. Conversely, the energy and materials sector exhibited minimal adoption of generative AI projects. Within business functions, sales and marketing departments led in generative AI deployments, while finance and procurement saw the least activity. Furthermore, AI was less frequently assigned to complex tasks like client management (only 10% of the time), with human intervention still preferred for routine tasks such as report summarization or email drafting (70% of the time).
The methodological approach and informal tone of the published report suggest that its primary aim may lean more towards promotional activities than rigorous academic or technological discourse. The report's authors advocate for strategic collaborations with experienced vendors to enhance the success rates of generative AI projects, a role NANDA itself is poised to fulfill. The paper concludes by emphasizing the "unprecedented opportunities" for vendors capable of delivering AI systems that are both learning-enabled and deeply integrated within an organization's existing infrastructure.
While the initial findings from the NANDA report might appear discouraging for those involved in generative AI deployment, the underlying motivations behind its publication slightly diminish its perceived objectivity. The recent fluctuations in stock prices could be partly attributed to such partisan analyses from entities with vested interests. However, it is perhaps more probable that the NANDA study merely echoes the prevailing concerns within financial circles regarding the practical utility and return on investment of generative AI as a core business instrument.
