Documentation

With the rapid proliferation of Internet-of-Things (IoT) and billions of connected devices, there is a paradigm shift taking place where big data is not only being processed in the core data center but also at the network edge. Field Programmable Gate Arrays (FPGAs), sitting at the intersection of performance and flexibility, are a promising solution for deep learning edge inference applications.

Artificial intelligence (AI) and machine learning (ML) technologies now power a rapidly expanding range of product and applications from deeply embedded systems to hyperscale data-center deployments. Although there is a huge degree of diversity in the hardware designs supporting these applications, all require hardware acceleration. Data orchestration encompasses the pre- and post-processing operations that ensure the data seen by a machine learning engine arrives at an optimal speed and in the most suitable form for efficient processing.

Thanks to the rise of big data and the rapid increase in computing power, machine learning technology has experienced revolutionary development in recent years. Machine learning tasks such as image classification, speech recognition, and natural language processing, operate on Euclidean data with a certain size, dimension, and an orderly arrangement. However, in many realistic scenarios, data is represented by complex non-Euclidean data such as graphs. In this context, many new graph-based machine learning algorithm, or graph neural networks (GNNs), are constantly emerging in academia and industry.

Deep learning's demand for computing power is growing at an incredible rate, accelerating recently from doubling every year to doubling every three months. Increasing the capacity of deep neural network (DNN) models has shown improvements across a wide range of areas ranging from natural language processing to image processing. This growth calls for the adoption of customized architectures that squeeze the greatest amount of performance out of each transistor available.

While the performance of an ASIC is typically high enough for broadcast-quality video processing, it supports only the feature set conceived of at design time and is not field upgradable. A CPU is the most flexible and easiest to design; however, clock frequencies have plateaued, and the era of dramatic improvements in performance are over. FPGAs represent a good balance between performance and flexibility for this class of applications.