India's AI Roadmap Prioritizes Small Models, But CPU Role Under Debate
India's strategic approach to artificial intelligence development is gaining clarity, with policymakers and government officials advocating for a bottom-up methodology. This strategy focuses on creating small, sector-specific AI models, shared infrastructure, and distributed access to computational resources. The Economic Survey 2026-27, alongside senior IT Ministry officials, has highlighted the potential of developing compact language models tailored to industries like healthcare, questioning the necessity for expensive Graphics Processing Units (GPUs).
Despite this shift, India continues to invest in GPU capacity under the IndiaAI Mission, which allocates Rs 10,372 crores over five years. The plan aims to triple installed GPU capacity from 38,000 to 100,000 units by the end of 2026. However, Hrishikesh Dewan, co-founder and CEO of Bengaluru-based Ziroh Labs, argues that small language models require less computational power than large frontier models, making them suitable for deployment on Central Processing Units (CPUs).
CPU Relevance in the AI Era: Efficiency and Operational Benefits
In an exclusive interview, Dewan explained the enduring relevance of CPUs. He noted that GPUs, originally add-ons for graphics-intensive tasks like gaming, have become dominant for compute-heavy jobs over the past decade. For instance, processing video often utilizes GPUs, while serving it to users relies on CPU-based servers. Since 2017, AI has driven demand for high computational power, with large models involving numerous equations requiring GPUs.
However, Dewan emphasized that not all AI applications need massive models. "If a model is designed to answer questions on a specific type of cancer, such as oral or breast cancer, it doesn't need to address philosophy," he stated. Small, domain-specific models have lower computational demands and can be executed efficiently on CPUs. Even when scaled nationwide, distributed loads can be managed with CPUs, avoiding GPU costs.
Agentic AI and Inference: CPU Capabilities Explored
Regarding AI agents, Dewan clarified that these applications typically use small models for straightforward tasks, such as booking flights. Agentic AI involves distributed intelligence from multiple sources, making CPUs well-suited for this distributed computation. On inference optimization, he explained that while TPUs and GPUs handle large models, CPUs are practical for small models when deployment scale and audience are considered.
Dewan identified key barriers to wider CPU adoption in AI: throughput and quality. Low token delivery speeds and inaccurate answers can frustrate users. Ziroh Labs' Kompact AI platform addresses these by doubling throughput without compromising quality, enabling models to run on cost-effective CPUs across cloud, on-premises, and on-device environments.
Kompact AI Development and Global Competitiveness
Dewan detailed Kompact AI's development, starting in 2021 with a focus on scientific advancements to scale AI models. By 2024, the engineering phase produced Kompact AI Runtime, which modifies equations for CPU efficiency, akin to Nvidia's CUDA for GPUs. The platform offers ready-to-use, fine-tuned models from sources like Hugging Face, requiring minimal coding for deployment.
Kompact AI utilizes CPUs globally, including from Indian data centers and major providers like Intel, AMD, Google Cloud, AWS, and Azure. Developers can choose hardware based on regulatory needs, such as keeping healthcare data in India. Dewan stressed that migration to CPU-based solutions incurs zero cost and reduces expenses compared to GPU usage.
On India's AI foothold, Dewan urged global competition, noting that resources like linguistic data are accessible worldwide. He praised initiatives like GPU-based data centers in space for potential energy savings, suggesting CPUs could offer even lower costs due to reduced power requirements.
In summary, while India's AI strategy champions small models, Dewan advocates for greater CPU integration to enhance affordability and efficiency, positioning the country competitively in the global AI landscape.
