CppNorth2022

With the slowing of Moore’s law and explosion of ML workloads - there has been a new renaissance in domain specific architectures (DSAs) to help meet today’s compute demands. A large swath of these architectures are spatial in nature where compute is often unrolled in space to expose as much parallelism to accelerate a given workload. With this comes a natural regularity to the overall architecture which software compilers can take advantage of. Here we explore the inherent challenges that traditional architectures like GPUs and CPUs have for the software compilation problem, and discuss how DSAs can avoid such problems in its software architecture. Additionally, due to the deterministic characteristics of many spatial architectures like FPGAs and Tensor Streaming Processors (TSPs) - we show how we can use software to abstract out many of the details of the hardware to make the software compiler agnostic to specific hardware parameters of the device (such as vector length or number of PEs). Unlike traditional architectures, this allows a software chip-model to drive the underlying algorithms mapping to a given architecture and accurately model the resulting performance of the hardware - even prior to the construction of the device. Furthermore, this makes migration to new devices much simpler and simplifies the overall software development process. We will illustrate these concepts using FPGA open source tools and Groq’s TSP.