AI model optimization chip developer GROQ receives $1.5 billion commitment from Saudi Arabia

Groq Inc. on Monday announcement Securing a $1.5 billion commitment from Saudi Arabia and expanding its AI chip offering to the country.

The transaction comes roughly six months after raising $640 million in funding from Samsung Electronics Co. Ltd., Cisco Investments and other supporters. The transaction valued GROQ at $2.8 billion.

Last year, Groq announcement An initiative to build an AI data centre in Dammam, Saudi Arabia. This facility is optimized for inference or tasks of performing neural networks during production after training. According to Reutersthe $1.5 billion commitment announced this week is directed towards expanding the data center.

The facility will be equipped with GROQ’s flagship LPU or language processing unit chips. The company says the processor is 10 times more energy efficient than the graphics processing unit. Furthermore, GROQ claims that programming LPUs is easy. This means deploying AI workloads on a chip requires less time and custom code.

Nvidia Corp.’s graphics cards can run not only large language models but also a variety of other workloads. In contrast, GROQ’s LPUs are optimized specifically for LLMS, one of the factors behind its efficiency. When engineers design chips that focus on narrow use cases, they can remove some of the components with more general purpose processors, such as GPUs.

Graphics cards break down AI processing tasks into simpler steps. Once the chip completes the steps, the hardware resources used to complete the calculation can be immediately reassigned to the next calculation. However, in practice, the process of reallocating hardware resources to workloads is often slowed down due to technical issues.

GROQ says that LPU streamlines processes. The chip has a mechanism that automatically determines the data that a particular set of circuits should process, how and where the output is transmitted. According to GROQ, this deployment will allow AI workloads to better utilize the on-chip computing resources of LPUs.

Another way that companies promise to increase efficiency is to improve chips in AI cluster exchange data.

LLM usually runs on several processors rather than one processor. To coordinate their work, those processors exchange data periodically. This is done with the help of a special networking chip. GROQ claims that LPU design reduces the need for external networking components, reduces costs, and makes AI clusters easier to drive CHIP.

The company ships LPUs with internally developed compilers. The compiler transforms the customer’s AI model into a format that is easier for the chip to handle. Along the way, optimizing these models to better use the underlying hardware. This is a task that developers usually need to perform manually.

Groq sells chips as part of an appliance called Groqrack. The system includes 8 servers, each with 8 LPUs working. The processor is linked by an internally developed real-scale called interconnects, which promises to remove the need for external switches.

One Groqrack can provide 12 PetaFlops performance when processing FP16 data points. This is commonly used in AI models to hold information. One Petaflop is equivalent to 10 billion computing operations per second.

GROQ makes chips available on a managed basis via a cloud platform called GroqCloud. The company updated its platform this week to allow customers to run workloads in new Dammam data centers.

Image: Groq