We have entered the next phase of the digital revolution in which the data center has stretched to the edge of the network and where myriad Internet of Things (IoT) devices gather and process data with the aid of artificial intelligence (AI).
As The Linux Foundation’s State of the Edge 2021 report related, we are in the midst of a transition to the “hyper-connected era” in which nearly every object can have computing and connectivity built in.
Edge devices in combination with AI and machine learning are seen as giving rise to the next Industrial Revolution, characterized by the decentralization of computing, communications, and business processes.
Hyperconnected networks, says PwC, are pushing closer and closer to ubiquitous connectivity. In this environment, localized IoT device-to-device communication can produce AI-driven insights and responses, when and where they are needed, without involving the cloud.
The Edge and Cloud Are One
As the hyperconnected network becomes ubiquitous, the lines between data center, cloud, and edge are blurring. This is the vision described by Jonas Boner, CEO and co-founder of Lightbend, in “When the Cloud and the Edge Are the Same Thing.”
Boner sees the rapidly evolving demands for intelligent low-latency processing of edge-generated data “creating requirements for a new approach combining cloud and edge into a single continuum.” A fully decentralized infrastructure, including the edge, he says, is a natural next step in the evolution of enterprise application architecture.
As computing becomes increasingly more distributed between the data center and the edge, says the State of the Edge 2022 report, a new “edge data center” model is emerging. While some cloud edge infrastructure might be in on-premises data centers, “more of it will be in new edge data centers, embedded in edge devices or even built right into the telecom infrastructure.”
Big data and the cloud are the key drivers behind the revolutionary movement to the edge. Every day, huge amounts of data are generated, streamed, and moved in cloud environments. From the smart refrigerator that knows you’re out of milk, to cloud gaming and on-demand movies, to patient monitoring systems in hospitals and retail inventory management systems, along with myriad other enterprise cloud applications, data is continually being created, transmitted, and analyzed.
Traditionally, to be understood, all this data had to be sent to a central cloud complex for analysis. But moving large amounts of data around can be costly and slow. And while some latency might be acceptable if you’re creating a monthly report, transmission delays are annoying if you’re streaming a movie, and can be a real problem if you’re using a point-of-sale system, performing surgery, or need to instantly understand a problem that’s slowing down production.
Edge computing reduces cost and latency by enabling data to be processed where it’s generated and consumed. Processing data locally instead of sending it to the cloud increases security, privacy, and reliability. It also enables organizations to more quickly scale up applications.
This is why organizations in sectors as diverse as healthcare, agriculture, retail, manufacturing, finance, energy, and government are deploying edge applications where they need visibility and faster data processing.
Smartenergy.com, for example, describes how edge computing is revolutionizing the power industry, replacing legacy control and monitoring systems and transforming the grid, in addition to enabling a new generation of smart applications that can leverage the latest advances in AI and machine learning.
Edge Is Where the Action Is
A massive movement to the edge can be seen in a variety of surveys. Among the key findings of “The State of Cloud Native Development 2021” is that edge computing has experienced rapid growth and has the highest adoption rate among developers across all surveyed sectors.
In 2018, the survey notes, only 10% of enterprise-generated data was created and
processed outside traditional centralized data centers or cloud services. By 2025, 75% of that data is expected to be created at the edge, in factories, hospitals, retail stores and cities, with much of it processed, stored, analyzed and acted on at the edge.
Gartner also sees the distributed enterprise driving computing to the edge. By 2025, says Gartner, more than 50% of enterprise-managed data will be created and processed outside the data center or cloud.
For this reason, Gartner advises infrastructure and operations leaders “to plan for the growth of IoT and related data at the edge, protect their enterprise from edge growing pains in security and connectivity, and prepare for changes in edge computing use cases.”
Similarly, the 2022 Red Hat Global Tech Outlook survey found that 61% of respondents ranked IoT, edge computing, or both as a priority emerging tech workload for 2022.
The massive movement to the edge has not been lost on major vendors like IBM, HP, Microsoft, Oracle, Amazon, Google, Dell, Red Hat, and Suse, all of which have introduced edge platforms, applications, and Kubernetes edge solutions.
The open-source community is generating a new generation of edge solutions under the umbrella of the Linux Foundation LF Edge program, The Open Infrastructure Foundation, The Edge Computing Group, and the Open-IX Association. Promising edge projects include EdgeX Foundry, Arkaino, Eve, Fledge, Open Horizon, and Baetyle.
Deloitte sees AI revolutionizing industries as diverse as health care, law, journalism, aerospace, and manufacturing, with the potential to profoundly affect how people live, work, and play.”
By automating tasks and processes and analyzing data on a scale previously impossible, and by using machine learning’s self-improvement capability to draw conclusions and make predictions, AI is improving performance and productivity for a wide range of enterprises. But many are finding the biggest gains from combining AI with edge computing for faster data processing and responses.
Embedding AI into IoT endpoints, gateways, and edge servers can significantly improve operational efficiency, customer service, and the speed of decision-making. It unleashes innovation and improves process optimization across industries, enabling timely understanding of customer data for personalization of apps and customer service, real-time automation in manufacturing environments, and rapid development and testing of data models. AI at the edge also can capture information humans miss in applications like video surveillance.
AI already provides the intelligence for self-checkout lanes and wearable devices, is helping banks run investment analyses, and is improving crop yields through IoT sensors in the field. AI is an underlying technology for cloud-based subscriptions for software, contact centers, platforms, and anything else available “as a service.” It is the intelligence behind recommendation engines and chatbots, and is the critical technology for making self-driving cars a reality.
AI also is a key element of the U.S. government’s modernization effort. As Fedtech Magazine reports, “Artificial Intelligence, machine learning and predictive analytics are increasingly viewed as a matter of national security.”
This concern is echoed in The National Security Commission on Artificial Intelligence (NSCAI) Final Report, which concludes that “AI technologies will be a source of enormous power for the companies and countries that harness them.”
Putting this strategy into action is the U.S. Defense Advanced Research Projects Agency (DARPA), which launched an Air Combat Evolution (ACE) project that uses AI in complex air combat maneuvering involving manned and unmanned aircraft.
New types of chips are needed for AI processing because the current generation of microprocessors have reached their performance limit. Computers today are gaining power not by increasing speed, but by adding more cores.
The edge is the biggest target for the new generation of AI chips, and PCIe cards are the hottest segment of the market, said Mike Demler, senior analyst with The Linley Group, which publishes the Microprocessor Report.
The need for greater AI processing power is reflected in the market projections. Allied Market Research (AMR) says the global artificial intelligence chip market size is expected to reach $194.90 billion by 2030, while Research and Markets sees the market growing to $304 billion by 2030.
Although the race to build AI chips is intense, no leader has emerged. There are various approaches to AI processing being developed, from tiny chips that run in small devices to large chips that run in servers.
“It’s impossible to keep track of all the companies jumping into the AI-chip space,” said Mike Demler “I’m not joking that we learn about a new one nearly every week.”
Among the new crop of AI chip companies are Adaptiva, Anari AI, Blaize, BrainChip, Cerebras Systems, Deep AI, EdgeCortix, Expedera, Flex Logix, Graphcore, Groq, Hailo Technologies, Kalray, Mythic AI, Roviero, SambaNova, Syntiant, TensTorrent, Thinci, and Untether AI,
The startups are competing against major players like Intel, IBM, Amazon, Advanced Micro Devices (AMD), NVIDIA, Arm, Alphabet (Google), Apple, NXP Semiconductors, Analog Devices, Samsung, Baidu, TSMC, LG, and Qualcomm.
Some of these established players are acquiring specialized AI chip companies to gain a competitive edge. Intel, for example, has purchased three AI chip companies: Nervana for $408 million in 2016, Habana Labs for $2 billion in December 2019, and Granulate for $650 million in March 2022.
Qualcomm acquired Nuvia for $1.4 billion in September 2021, while AMD purchased Xilinx for $35 billion in October 2020. NVIDIA announced the purchase of Arm for $40 billion in September 2020, but abandoned the deal in February 2022.
IBM is investing heavily in AI chip research and development. In 2019, IBM announced that it was investing $2 billion in a new artificial intelligence research hub in New York, including the AI Hardware Center at the SUNY Polytechnic Institute in Albany. IBM’s research initiative has borne fruit in the form of the Telum Processor, unveiled last August, which is designed for heavy-duty AI-specific workloads like fraud detection, loan processing, clearing and settlement of trades, anti-money laundering, and risk analysis.
Just recently, IBM introduced the Artificial Intelligence Unit (AIU), a specialized processor based on Telum that it describes as a complete system-on-chip, with 32 processing cores and 23 billion transistors, that can be plugged into any computer or server with a PCIe slot.
Google, which has advanced AI development through its TensorFlow software, has introduced a Tensor Processsing Unit (TPU) AI chip that can handle trillions of operations per second, is energy efficient, and can generate complex chip designs in just hours.
AI at the Edge Is the Biggest Piece of the Pie
As analyst George Anadiotis notes, “AI is the most disruptive technology of our lifetimes, and AI chips are the most disruptive infrastructure for AI.”
AI chips at the edge will account for the lion’s share of this disruption, according to analysts. Venturebeat reports that edge computing is expected to make up roughly three-fourths of the total AI chipset business in the next six years. Semiconductor Engineering cites projections that AI sales will grow rapidly to the tens of billions of dollars by the mid 2020s, with most of the growth in edge AI inference.
Gartner projects that by 2025, multiple social, business and technology trends will propel edge computing and AI at the edge to become mainstream, necessary parts of IT architectures.
Kubernetes Powers the Edge
Building edge AI applications to fulfill the needs of all the use cases is difficult. There are many forms of data that must be handled across multiple steps, the apps need to run across a variety of distributed platforms, and the apps must be kept current, often in situations that require continuous updating.
Many edge AI developers are turning to containers to increase efficiency, automate workflows, speed deployment and updates, and improve scalability and security. Most are using cloud-native, open-source Kubernetes to orchestrate their containers.
The State of Native Cloud Development 2021 report found that 76% of developers working on edge computing applications were using containers and 63% were using Kubernetes.
Using Kubernetes speeds deployment of new applications by a factor of 10 times or more and makes the development, packaging, and deployment process predictable and consistent. It enables AI to run across different platforms, toolsets, and chipsets, and provides for continuous improvement through exponentially faster, large-scale AI updates. Kubernetes can also optimize workload placement to improve edge AI performance.
In a survey sponsored by D2iQ and conducted by Vanson Bourne, 40% of respondents said AI and machine learning were their most popular workloads for Kubernetes, and 88% agreed that in the next two years Kubernetes would be the platform of choice for running AI and machine learning workloads.
“The Future Is Smart: Cloud Native + AI” describes how the ability to harness the power of AI will give businesses a competitive edge and why Kubernetes and AI are a perfect match.
Emerging Trends for AI at the Edge
Over the next few years, we can expect to see the use cases for AI at the edge expand substantially, driven by a number of key trends, including the following.
Edge-native applications. Most of today’s enterprise edge use cases are primarily cloud use cases adapted to take advantage of the speed and cost savings of edge processing. Edge-native applications are built to run on distributed networks. They can respond to streaming data as it happens, make changes on the fly, and can migrate application logic to other edge locations depending on environmental conditions or the movement of assets.
In the enterprise, they increase the reliability of the edge, enable seamless application mobility, increase security and privacy protection, and further reduce latency. Development has begun on these applications, but the Linux Foundation believes long-term demand depends on the maturation of key technologies, such as augmented and virtual reality, and autonomous systems, such as those for closed loop enterprise IT functions.
5G. With its ability to deliver network speeds up to 100 times faster than 4G with minimal latency and increased security, reliability, and efficiency, 5G combined with AI will power new innovation at the edge and help bring the fulfillment of IoT closer to the potential. According to the MIT Technology Review, “The combination of AI and 5G will transform the enterprise and accelerate economic growth, as 5G networks provide the backbone, scalable bandwidth, and remote compute resources to process increasing volumes of data that will fuel the proliferation of AI.”
Fog computing. Fog computing is a compute layer between the cloud and the edge. Instead of sending huge streams of data directly to the cloud, the fog layer analyzes the information and only passes a priority subset of data to the cloud, reducing traffic and saving storage space. In this way, fog computing provides a location for AI processing between the edge and the cloud. It also enables greater scalability and the analysis of data from multiple edge locations, for example, processing data from road sensors across a city and adjusting traffic lights to keep cars moving.
Augmented and virtual reality. Augmented reality (AR) presents digital information visually and in real time,while virtual reality (VR) enables the creation of artificial environments. Some early uses of these technologies include realistic training for employees, immersive sales and marketing experiences, the ability for consumers to virtually try on makeup and clothes, inspect a house, and see a piece of furniture in their room, as well as driver assistance and improved industrial design applications.
AR and VR applications are bandwidth-hungry and generate massive amounts of data, so they will benefit from the reduced latency and data congestion and improved mobility of 5G at the edge. Markets and Markets projections show a huge opportunity for enterprise AR and VR applications, with the market growing from $37 billion in 2022 to $114.5 billion in 2027.
Welcome to the Hyperconnected Era
Digital technology has advanced at a dizzying pace through a series of disruptive technologies, including the microprocessor, PC, Internet, mobile devices, cloud computing, AI, machine learning, edge, and IoT.
Now we see the data center, cloud, and edge converging into an interconnected whole. As edge and AI technologies mature over the coming years, they will bring us closer to a world in which business innovation rivals that of the e-business revolution unleashed by the Internet in the 1990s.
AI at the Edge with D2iQ
D2iQ enables easy enterprise-level edge deployment and management through the D2iQ Kubernetes Platform and Edge/IoT add-on solution.
D2iQ customers who are deploying DKP at the edge include BMW, GE Healthcare, Royal Caribbean Cruise Lines, Koch Industries, U.S. Navy, Department of Homeland Security, and BP Security.
