Why decentralization matters
Some periods in the history of computing have led to advances in centralized computing while other eras have witnessed the development and adoption of decentralized architectures.
In the early days of computing, there were few users with bulk data processing requirements. This favored a centralized infrastructure. In the 1980s that changed, as microcomputers gained favor and new kinds of software, like spreadsheets, found a broad user base. More recently, cloud computing demonstrated the advantages of centralized infrastructure with on-demand, scalable computing and storage.
But today, advances in mobile communications and an increase in the processing power of edge devices together are creating opportunities for a more decentralized, distributed computing infrastructure to complement cloud computing.
The state of cloud and edge computing
In today’s mobile and cloud computing environment, most of the compute and storage resources reside in centralized clouds. Meanwhile, at the edge of this environment, applications on mobile and Internet of Things (IoT)-connected devices transmit data to the cloud. For many use cases, this model works quite well.
If edge devices are primarily used for data collection and do not require autonomous control or fast response times, then transmitting data to a centralized processing and storage infrastructure isn’t a problem.
For example, a fleet of vehicles using sensors to monitor fuel consumption can send data to a centralized computing infrastructure for long-term planning, budgeting, and fleet optimization. Centralizing data collection and analysis is a reasonable approach for this use case.
But if that same fleet needs to use safety monitoring equipment to detect oncoming vehicles and prevent a collision, they need to analyze and react in short periods of time. Reliable collision protection systems cannot depend on having reliable, low-latency connections to a centralized server. It’s more appropriate to move that analysis and decision-making to the device itself.
To understand the benefits of an emerging decentralized architecture built around computing at the edge, it helps to discern the role of three major elements of the architecture: telecommunications infrastructure, edge devices, and cloud resources.
Advances in telecommunications, like the widespread use of 5G networks, will bring lower latency and higher capacity bandwidth. A growing number of mobile and IoT devices that will be deployed in the next several years will take advantage of the capabilities of these networks. However, this new technology alone is not enough to meet the growing need for more connectivity, computation, and storage.
Some carriers are adding processing and storage capacity at mobile base stations. This can absorb some of the new demand for computation and storage but cannot accommodate growing volume and velocity of data generated at the edge. Gartner estimates that today 10 percent of all data is created outside the cloud or data center. That number is expected to grow to 50 percent by 2022.
Edge computing devices
Today’s edge devices are no longer just data conduits — they are computing nodes in a highly distributed system. Edge devices still communicate with centralized servers in the cloud, but instead of transmitting all or most of their data to a centralized repository, those devices may send samples, summary data, or anomalous data only.
Since edge devices are capable of analyzing data locally, they are becoming more capable of autonomous behavior. The confluence of advances in machine learning and artificial intelligence (AI), combined with the increased computing capacity of devices at the edge enabling low-latency, intelligent processing without requiring additional resources in the cloud.
Driver-assisted and autonomous vehicles have moved from research tracks to public roadways. Tesla’s autopilot, for example, enables cars to learn from data collected from sensors deployed in other Tesla vehicles. IoT devices in homes, such as intelligent thermostats, can learn the patterns of occupants and adapt to their preferences and schedules.
The Tesla example also demonstrates the value of centralized computing coupled with edge computing. Cars can learn individually, but they also share what they know with other intelligent cars.
This type of use case does not require low latency. An improved machine learning model can be uploaded and distributed to other vehicles over an extended period of time without adversely affecting vehicle operations. Vehicles are able to transmit highly refined information, such as a machine learning model, instead of all the raw data that was used to train the model.
In 2006, Amazon introduced Amazon Web Services with the release of AWS S3, a scalable object store. Since then, Amazon has added a suite of compute, storage, networking, and specialized application services, including IoT and machine learning services. Other cloud vendors include Google, Microsoft, and IBM.
Brian Nowak, cloud analyst with Morgan Stanley, noted that 2017 marked an inflection point for cloud computing. At that point, 20 percent of computing workloads were running in the cloud.
Growth is not projected to slow. In fact, one significant driver of growth would be new applications, including IoT services. This growing demand is an opportunity, but one that still has significant challenges.
Gartner estimates there will be 25 billion connected devices by 2020. The global collection of devices will generate 1.7 megabytes of data per second for every human on the planet. This volume and velocity of data could overwhelm both cloud resources and telecommunications infrastructure. In addition, many of the new devices will operate in environments with limited or unreliable connectivity. This scenario requires decentralized computing and storage.
In addition to these technical drivers to decentralization, increasing concerns about privacy and security are shaping architectural decisions as well.
Non-technical drivers of decentralization
Consumers are becoming increasingly wary of large cloud platforms collecting, storing, and monetizing their data.
In 2014, a researcher developed a personality quiz application that collected data about each users social network, including detailed data about their friends. This data was extracted from Facebook’s systems and stored in the researcher’s own database.Cambridge Analytica, a political data firm, acquired the misappropriate data.
When details of the misappropriation surfaced, the response was swift and negative. The U.S. Federal Trade Commission opened an investigation. The U.S. Congress and U.K. Parliament raised questions and held hearings on the incident. Attorneys General in several U.S. states also sought details about the misappropriated data.
The backlash against Facebook’s failure to protect personal information went beyond these government responses. Non-tech publications, such as The Guardian, Slate, and Business Insider advised readers on how to delete their Facebook data. The #deletefacebook campaign became a global, grassroots movement advocating for users to stop using the social network.
Shortly after the Cambridge Analytica story broke, users discovered that Facebook collected call and messaging data from Android smartphones. This does not have to happen. Device manufacturers can implement controls to protect customer privacy at the edge.
Enterprises are witnessing the failure of third parties to protect their customers’ data. As GDPR goes live in Europe and with calls for similar legislation in the United States, businesses have a responsibility to protect their customers’ data.
At the same time, consumer attitudes about privacy are changing. A recent survey found only 20 percent of respondents fully trusted companies to keep their data safe but 77 percent consider how much they trust an organization before making a purchase from them.
Enterprises must address the risk of relying on cloud platforms operated by third-party intermediaries to protect their data. One way to mitigate the risk of a centralized data breach is to minimize the amount of data stored in the cloud.
Edge computing, along with encryption and sound privacy controls, can help reduce the impact of a data breach in or misappropriated data from a centralized cloud.
Cloud and edge combine to optimize the user experience
Increased decentralization does not reduce the importance of cloud computing, but it does change how applications deliver services to users.
The cloud has the advantage of large-scale computing and storage capabilities that is ideal for large volume data analysis workloads. Edge computing is closer to the user and best suited for analyzing the local environment, responding rapidly, and enabling autonomous decision making at the device.
Information that is useful beyond the device, such as a newly learned machine learning model, can be distributed to other devices through the cloud. Information that is sensitive, like an end user’s private data, can be kept at the edge to reduce the risk of a large scale data breach.
Both cloud and edge computing will continue to grow and they will continue to become more integrated as developers learn to take advantage of the complementary strengths.