The world’s attention is being grabbed by artificial-intelligence (AI) technology becoming pervasive in consumer electronics, but that’s only half of the edge AI story. Edge AI can also be scaled up to be as computationally powerful as needed. 

Powerful edge AI servers have become indispensable tools in the manufacturing sector, improving production processes, increasing manufacturing line efficiency, providing superior inspection, and monitoring of products, plus more. 

Advantages of Edge AI 

All AI models require some training. The process is particularly elaborate for the most comprehensive models, which require repetitively running through enormous datasets until they can reliably produce expected results in response to queries. Training complex models such as large language models (LLMs) and vision language models (VLMs) almost always take the formidable processing capabilities and memory resources resident only in giant data centers. 

After being trained, most of the work that AIs perform is called inference. This includes parsing new inputs in the context of what the AI has already learned. Inference doesn’t need quite the processing power of training. Improving the capabilities of an application or adding features typically requires AI models that are more complex, though. As products become more sophisticated, some inference workloads turn quite compute-intensive.  

Many of the innovations in AI in the last few years have involved getting processors to run inference workloads far more efficiently in terms of both processing power and the energy consumed. This has made it eminently practical to run more AI workloads — and more sophisticated workloads — in servers located at the network edge, and even in smaller consumer electronics such as smart speakers and nanny cams. 

Edge AI is faster and more efficient than relying on the cloud. “Edge computing provides notable energy efficiency benefits compared to large data centers by reducing data transport, lowering latency, and enabling dynamic resource allocation,” according to “Edge AI for Internet of Energy: Challenges and Perspectives,” published in the journal Engineering Cyber Physical Human Systems Internet of Things; Engineering Cyber Physical Human Systems. 

Latency can be significantly lowered by not sending data on a round trip to and from some distant computing center, which results in achieving faster results. For example, GE Healthcare reported notably faster results after embedding AI in its X-ray systems. 

Edge AI can be more efficient in several ways. It minimizes network bandwidth usage, again by not shipping data to and from a data center. This can reduce telecom service bills, and the energy not expended on data transmission is saved.  

Generally speaking, edge AI applications tend to run smaller, more specialized AI models, which means an AI processor at the edge is likely to consume less power than a server running AI workloads in a data center.  

There are applications at the edge, including some in manufacturing, that don’t need to run continuously. In these instances, edge AI can operate in standby and/or sleep modes to minimize energy consumption. While engineers must balance the factors involved in any specific application, it’s possible to achieve significant efficiencies with edge AI.  

Finally, data generated and used at the edge stays at the edge, serving the interests of both data security and data privacy. This is often a decisive consideration for companies seeking to protect their intellectual property (IP), or companies operating with legal obligations to protect consumer information, e.g., medical facilities.  

Edge AI in Manufacturing 

Edge AI is a powerful means for meeting a variety of manufacturers’ needs and improving a range of their business operations and practices. Edge AI can perform data analysis in real-time, which facilitates rapid decision-making, enhances operational agility, and streamlines workflows, ultimately driving additional efficiency gains, cost savings, and improved product quality. By analyzing real-time data, AIs are able to identify conditions and trends. Manufacturers can enable AI systems to trigger automatic responses in production lines, trip safety alarms, and more. 

Machine vision has become integral for any number of manufacturing applications, from quality assurance to process optimization to waste reduction. Production lines vary: Some might inspect only a sample of parts for defects, perhaps only 1 out of every 20, or 1 out of every 5. The choice not to inspect every part can sometimes depend on limitations inherent in the inspection of equipment. 

However, by using computer vision backed by edge AI, manufacturers can inspect every part coming off the production line. Even in instances where every part is already being inspected, AI-based systems have proven to be better at detecting defects. 

Machine-vision systems backed by AI quickly identify defects (Fig. 1). The AI will immediately evaluate the number and severity of defects that appear and can be programmed to recommend taking remedial action to minimize waste, or to automatically take action.

In fact, it’s possible for machine-vision systems backed by edge AI to analyze variations in product quality that indicate an incipient problem. In other words, AI can be trained to identify variations in quality and alert line managers to take remedial action before the appearance of actual defects. All of this ultimately leads to minimized production line downtime. 

By some estimates, leveraging AI in manufacturing may reduce machine downtime by 30% to 50%, and quality-related costs can be lowered by 10% to 20%.   

Such predictive capabilities could also be profitably used in the maintenance of production equipment. Without AI, manufacturers must schedule regular maintenance checks. If the maintenance check reveals no problems, then the scheduled shutdown has wasted production time.  

Meanwhile, if maintenance is required between scheduled checks, without AI monitoring, the need may go unidentified, leading to downtime and possibly expensive equipment damage. AI is singularly capable of continuously monitoring equipment and learning what performance anomalies are indicative of pending equipment failure. Edge AIs can be relied on to recommend maintenance checks when necessary, and only if necessary. This minimizes costly downtime and increases operational reliability and efficiency.  

One of the manufacturing industry’s most important and expensive challenges is workplace safety. Edge AI is invaluable for analyzing data from security cameras to identify potentially dangerous behaviors or situations. Companies also use edge AI to connect data feeds from CCTV, machinery, and operational software to provide real-time predictive analysis of potential hazards. AI models could, for example, identify when an employee isn’t wearing protective gear or is moving too close to dangerous machinery. 

Furthermore, as autonomous mobile systems are increasingly being introduced in workspaces, AI will be instrumental in avoiding hazardous situations. Real-time processing of such large amounts of data is practical only with edge AI. Since most manufacturing facilities already employ video monitoring systems, this safety boost is a matter of utilizing that data most effectively. 

Optimized AI Solutions 

There’s a thriving market for edge AI servers. Manufacturers should be careful to evaluate servers engineered for long-term reliability. Companies commonly maintain the computing infrastructure in manufacturing operations for far longer than, say, desktop computers. While processing capabilities and operational efficiencies are key factors when considering the installation of edge AI systems, there are others. 

One is the mean time between failure (MTBF), which speaks to system reliability. It’s also important to evaluate if the physical environment where the edge AI server(s) will be installed requires additional ruggedization. This might include the ability to operate across wider temperature ranges, or the ability to withstand shock and vibration. Fanless servers are indicated for factories and other work environments where dust or other particulate matter is commonly present. 

The main challenge for manufacturers is specifying a configuration of edge AI infrastructure that’s optimized for the mix of workloads that must be handled. Manufacturers can now choose from among a wide selection of standard servers integrating any of a range of processors and AI accelerators to perform AI workloads. 

Some of the more established suppliers of edge AI servers have modular architectures, which give manufacturers the flexibility to build semi-custom edge AI server systems to fulfill specific requirements. Such requirements can include anything from processing capabilities to level of ruggedization to the physical dimensions of the system being installed. Importantly, a modular approach also leaves open paths for upgradability over the course of years. 

Manufacturers planning to develop their own applications should also inquire about the availability of software development kits (SDKs). A useful SDK will already contain modules that manufacturers can use so that they don’t have to build applications from scratch. Examples include applications that can visually inspect products and need only to be modified to monitor printed circuit boards, the seals on containers, or anything else. This makes AI development easier and reduces time-to-market. 

For instance, Supermicro has a modular approach to building servers that’s resulted in a very broad portfolio, including a wide selection of standard products for both the cloud and the network edge (Fig. 2). This modular approach also makes it easier for the company to devise optimized solutions. Supermicro’s edge AI solutions are able to bring the capabilities of AI accelerators to ruggedized, remote environments such as factory floors, where they can be integrated into key business processes.

Successfully deploying and running AI applications at the edge often requires specialized hardware. AI accelerators are the workhorses in this model, processing large amounts of data based on a pre-trained model. 

NVIDIA is the global market leader for these accelerators, with a broad range of AI-optimized CPU and GPU modules for workloads of any type and scale. The NVIDIA Jetson Orin NX platform for the embedded edge, NVIDIA RTX PRO 6000 Blackwell Server Edition, and NVIDIA H200 NVL GPUs for enterprise edge all have distinctive use cases based on the type and volume of data being processed, as well as deployment factors such as size and power consumption. Selecting the right hardware, therefore, includes identifying which AI accelerator is required and ensuring that the server platform is compatible. 

Developing AI applications can be a daunting and costly process. That’s where a platform like NVIDIA Metropolis comes in. This end-to-end platform helps businesses fast-track the creation and deployment of intelligent video analytics by offering pre-trained models, SDKs, and optimized infrastructure. For factories, this means faster rollout of applications that save costs, improve production throughput, and make factories safer, without starting from scratch. 

By simplifying AI development and reducing the time-to-market, NVIDIA Metropolis helps company’s lower costs while unlocking powerful insights from their video data, turning everyday surveillance into a strategic business asset.