Caixin News Agency, September 30th (Reporter Fu Jing) Currently, the number of large-scale models in China with a scale of over 1 billion parameters has exceeded 100. Although the technology is developing vigorously, the actual implementation level still needs to be improved. Recently, at the 2024 China Computing Power Conference held in Zhengzhou, Caixin reporters observed that the application of large models still attracts attention from the industry chain. All-in-one machines have become a hot trend in the commercial exploration of large-scale models, and the computing hardware manufacturer Inspru Electronic Information (000977.SZ) also launched the Yuan Brain Enterprise Smart EPAI (Enterprise Platform of AI) all-in-one machine at this conference, providing a complete solution for large models including "computing power + platform + services."

Wei Jian, General Manager of the Product Solutions Development Department of Inspur Electronic Information, stated in an interview with Caixin and other media, "There is a gap between the current application status and market trends. How to quickly implement large-scale models in enterprises, especially traditional enterprises, based on this background, we have developed the all-in-one machine for large-scale models."

Earlier data from the Ministry of Industry and Information Technology's CCID Research Institute showed that in 2023, China added 368 AI companies, and the adoption rate of generative AI companies has reached 15%. Among them, the adoption rates in the manufacturing, retail, telecommunications, and medical industries were 5%, 13%, 10%, and 7% respectively. It is estimated that by 2035, generative AI is expected to contribute nearly 90 trillion yuan in economic value globally, with China exceeding 30 trillion yuan, accounting for over 40%.

Wei Jian also mentioned a set of data in the interview: In AI market public tender projects, the proportions of hardware were about 60% last year and 61% in the first half of this year, software accounted for about 11% and 5% respectively, and the proportion of services in the first half of this year increased from 17% last year to over 30%. "It means that hardware and services account for more than 90% of the total project amount."

Wei Jian stated that the all-in-one machine released by Inspur Electronic Information this time mainly targets customers in industries such as manufacturing, finance, traditional ISVs, and SIs. For example, for SIs, whether it is multi-model management or multi-element computing power management, there is still a need for specialized tuning capabilities for large models, and we can integrate services into the all-in-one machine product.

According to reports, the Yuan Brain Enterprise Smart EPAI all-in-one machine is based on the Yuan Brain server designed for large model application scenarios, pre-installed with the Yuan Brain Enterprise Smart EPAI Enterprise Large Model Development Platform, supporting multiple computing powers with 8 NVIDIA's latest Hopper architecture GPUs fully interconnected, pre-installed with 7 basic large models such as Yuan2.0, Baichuan2, ChatGLM3, Qwen1.5, GLM4, Llama3, aimed at solving problems related to data processing, model fine-tuning, RAG building, model deployment, application launching, and system operation and maintenance.

Caixin News Agency reporters noted that the Yuan Brain Enterprise Smart EPAI all-in-one machine covers five product specifications and mainly targets scenarios such as inference, integrated training and inference, and cluster delivery of whole machine cabinets.

"Inference computing power is a very important growth direction, especially edge inference. Whether it's vehicle-road coordination, smart parks, or highway toll booths, the inference applications in the edge field focus on different product forms," Wei Jian told Caixin journalists.

"In fact, large models go through roughly three stages from pre-training, fine-tuning to inference. The industry's current focus has shifted to fine-tuning and inference, with the increasing importance of fine-tuning becoming more prominent," Owen ZHU, an AI application architect at inspur electronic information industry, said in an interview.

Owen ZHU stated that the all-in-one machine released this time also integrated some fine-tuning technologies that are currently matched with computing power and are relatively easy to use in the industry. He gave an example, "A 10 billion parameter model may require 200-300GB of memory for fine-tuning, one machine may be a bit 'stretched,' not to mention that we are now using 30B, 40B models, at least three or four machines are needed, and the threshold is a bit too high. Some efficient fine-tuning techniques can fine-tune a 10B parameter model with less than 10GB of memory. That is to say, the application of some new technologies can reduce the threshold of computing power."

From a cost perspective, Wei Jian told Caixin journalists that the price of inspur electronic information industry's large model all-in-one machine ranges from two to three hundred thousand to two million. "If users have an initial investment of around 0.5 million in hardware devices, they can train models ranging from at least 1 billion to 30 billion parameters, and use better computing power on a single machine."

At the same time, she stated that inspur electronic information industry's quote to users is for the overall quote of the all-in-one machine, but "if you dismantle the all-in-one machine, hardware may account for 80%, and software 20%. We are a product company, doing this business model, it's more about stimulating the application of the entire industry ecosystem."

"The business logic of the all-in-one machine is correct, whether it is recognized by customers is crucial," an AI computing power practitioner told Caixin journalists.

Caixin reporter asked about the current user's willingness to accept large-scale integrated machines. Wei Jian said, "I understand that the willingness to accept is quite strong, and the integrated machine is quite compatible with the usage habits of some traditional hardware users. In the past three to four months, we have conducted POC tests with nearly a hundred companies, and about 30% of users are willing to do application customization development, let us organize data, etc."

It is worth noting that the popularity of large-scale integrated machines in the industry is high. Mainstream AI large-scale model manufacturers such as Zhipu AI, Sensetime, and Pactera International, as well as ICT service providers and ISV service providers, have all launched related integrated machine products.

Owen ZHU said, "Everyone is talking about integrated machines, but in fact, the connotation differences are quite significant. Perhaps when we first heard about the concept of integrated machines, it was about training and promoting integration, but now what we are talking about is packaging the hardware with solutions oriented towards large-scale models, it is a development platform for large models."

"Some large model manufacturers are actually partners with us, but our integrated machines are different from theirs. Large model manufacturers are strong in software and algorithm capabilities. We emphasize more on computing power scheduling management, algorithm scheduling, adapting to different models, including a lot of research on model parameters, actually, in order to better leverage the advantages of hardware performance." Wei Jian told the Caixin reporter.