Huawei has released CloudMatrix 384 AI chip cluster, a new system for AI learning. It employs a cluster of Ascend 910C processors coupled via optical links. The distributed architecture means that even though individual Ascend chips are no more powerful than competitors’ chips, the system can still outperform traditional hardware GPU setups, especially in terms of resource usage and on-chip time.
Huawei’s new framework positions the company as a “formidable challenger to Nvidia’s market-leading position, despite continued U.S. sanctions,” the company claims.
To use the new Huawei framework for AI, data engineers will need to adapt their workflows using frameworks that support Huawei’s Ascend processors, such as MindSpore, available from Huawei and its partners.
Framework migration: PyTorch/TensorFlow to MindSpore
Unlike NVIDIA’s ecosystem, which primarily uses frameworks like PyTorch and TensorFlow (designed to take full advantage of CUDA), Huawei’s Ascend processors perform best when used with MindSpore, a deep learning framework developed by the company.
If your data engineer has already built a model in PyTorch or TensorFlow, you may need to convert the model to MindSpore format or retrain it using the MindSpore API.
Note that MindSpore uses a different syntax, training pipeline, and function calls than PyTorch and TensorFlow, so replicating the model architecture and training pipeline results will require some re-engineering. Individual operators behave differently, for example, padding modes for convolutional and pooling layers. There are also differences in how the default weights are initialized.
Use MindIR for model deployment
MindSpore employs MindIR (MindSpore Intermediate Representation), which is very similar to Nvidia NIM. According to MindSpore’s official documentation, once a model is trained in MindSpore, it can be exported using the Mindspore.export utility, which converts the trained network to MindIR format.
As detailed in DeepWiki’s guide, deploying a model for inference typically requires loading an exported MindIR model and performing predictions using MindSpore’s inference API for Ascend chips, which handles model deserialization, assignment, and execution.
MindSpore separates training and inference logic more clearly than PyTorch or TensorFlow. Therefore, all preprocessing must match the training input, and the execution of static graphs must be optimized. For additional hardware-specific tuning, we recommend MindSpore Lite or Ascend Model Zoo.
Adaptation to CANN (Computing Architecture for Neural Networks)
Huawei’s CANN has a set of tools and libraries tailored for Ascend software and is functionally equivalent to NVIDIA’s CUDA. Huawei recommends using CANN’s profiling and debugging tools to monitor and improve the performance of models on Ascend hardware.
Execution modes: GRAPH_MODE and PYNATIVE_MODE
MindSpore offers two execution modes.
- graph mode – Compile computational graphs before execution. This allows the graph to be analyzed during compilation, resulting in faster execution and better performance optimization.
- PYNATIVE_MODE – Immediate execution of operations makes the debugging process simpler, and more detailed error tracking makes it suitable for early stages of model development.
For initial development, PYNATIVE_MODE is recommended for simpler iterative testing and debugging. When you are ready to deploy your model, switch to GRAPH_MODE to achieve maximum efficiency with Ascend hardware. By switching between modes, engineering teams can balance development flexibility with deployment performance.
The code must be adjusted for each mode. For example, for GRAPH_MODE, it is best to avoid Python native control flow whenever possible.
Installation environment: Huawei ModelArts
As you might expect, Huawei’s ModelArts, the company’s cloud-based AI development and deployment platform, is tightly integrated with Huawei’s Ascend hardware and MindSpore framework. It’s comparable to platforms like AWS SageMaker and Google Vertex AI, but optimized for Huawei’s AI processors.
According to Huawei, ModelArts supports the entire pipeline, from data labeling and preprocessing to model training, deployment, and monitoring. Each stage of the pipeline is available via an API or web interface.
In summary
Adapting to MindSpore and CANN may require training and time, especially for teams accustomed to the NVIDIA ecosystem, and data engineers will need to understand a variety of new processes. These include learning how CANN handles model compilation and optimization for Ascend hardware, adjusting tools and automation pipelines originally designed for NVIDIA GPUs, and new APIs and workflows specific to MindSpore.
Although Huawei’s tools are evolving, they lack the maturity, stability, and broad ecosystem support that frameworks like PyTorch with CUDA provide. But Huawei hopes the move to its own processes and infrastructure will eventually pay off, allowing the organization to reduce its dependence on U.S.-based Nvidia.
Huawei’s Ascend processors may be powerful and designed for AI workloads, but they have limited distribution in some countries. Teams outside of Huawei’s core markets may struggle to test or deploy models on Ascend hardware unless they use a partner platform like ModelArts that provides remote access.
Fortunately, Huawei provides extensive migration guides, support, and resources to help you with any migration.
(Image source: “Huawei P9” in 405 Mi16 is licensed under CC BY-NC-ND 2.0.)
Want to learn more about AI and big data from industry leaders? Check out the AI & Big Data Expo in Amsterdam, California, and London. This comprehensive event is part of TechEx and co-located with other major technology events. Click here for more information.
AI News is brought to you by TechForge Media. Learn about other upcoming enterprise technology events and webinars.
