Trust in AI deep dive:Sustainable
As AI evolves and becomes more widespread, the power consumption and costs associated with computational processing are increasing exponentially. This is a critical business challenge that no company can afford to ignore, from the perspectives of both sustainability and financial discipline. To continue relying on AI, sustainability—the ability to operate smoothly over the long term—is essential.
Featured Content: A Journey Through History—Learning from the Past
Events that marked turning points in history raise questions that still resonate with today’s business leaders: the heliocentric theory, the Renaissance, the Titanic, the Silk Road, and the Age of Discovery. This series uses stories from the past as clues to understand the meaning of “trust” in the age of AI.
“1-bit Quantization Technology” that makes advanced AI lightweight, low-cost, and easy to use
Technology introduction
1-bit quantization technology is a technique that minimizes the size of AI models while preserving as much of their capability as possible. AI models typically require significant amounts of memory and computational resources, but this technology enables generative AI to run faster, more affordably, and with lower power consumption. Furthermore, it makes it easier to use generative AI in field deployments or on-premises environments without the need for high-performance GPUs.
Why it matters
The value this technology brings to businesses is the ability to use advanced AI with lower costs and fewer constraints. It makes it easier to use generative AI even in environments with limited computational resources—such as on-premises environments, factory equipment, store terminals, and edge devices—and is expected to reduce initial investment and operational costs, reduce power consumption, and lower barriers to production deployment.
Example use case
Reducing the cost and power demands of generative AI
The technology demonstrated up to 94% memory reduction and enabled large generative AI models that previously required four GPUs to run on a single GPU, opening opportunities for deployment on smartphones, industrial equipment, and other edge devices.
“AI Computing Broker,” middleware for efficient GPU utilization
Technology introduction
AI Computing Broker (ACB) is middleware designed to efficiently share the GPUs required for AI processing and make effective use of limited computing resources. While GPUs are indispensable for AI training and inference, in practice, there are often periods before and after processing when GPUs are not in use. ACB increases utilization by leveraging this idle time to flexibly allocate GPUs to other tasks. A key feature is its ability to facilitate the operation of multiple AI applications with fewer GPUs.
