Today’s high-end system-on-chips (SoCs) rely heavily on sophisticated network-on-chip (NoC) technology to achieve performance and scalability. As the demands of artificial intelligence (AI), high-performance computing (HPC), and other compute-intensive applications continue to evolve, designing the next generation of SoCs will require even smarter and more efficient NoC solutions to meet these challenges.
Although these advancements present exciting opportunities, they also bring significant hurdles. SoC designers face rapid expansion in architecture, time-to-market pressures, scarcity of expertise, suboptimal utilization of resources, and disparate toolchains.
Exponential growth in SoC complexity
SoC designs have reached unprecedented levels of complexity, driven by advancements in process technologies and design tools. Now, SoCs typically include between 50 and 500+ IP blocks, ranging from processor cores and memory controllers to specialized accelerators for AI and graphics.
These blocks, which once contained just tens of thousands of transistors, now house anywhere from 1 million to over 1 billion transistors each. As a result, these SoCs incorporate a staggering total of 1 billion to over 100 billion transistors, reflecting the exponential growth in both scale and sophistication, as shown in the figure below.
The above chart highlights relationship between increasing transistor budgets and use of SIP blocks. Source: Arteris, based on and
This growth in IP blocks and transistor density has enabled the development of advanced architectures featuring multiple processor clusters. Each cluster typically contains up to 8 or more cores in mainstream designs, with high-performance configurations reaching 32 or more cores.
Today, these clusters are organized into arrays to provide massive parallelism. These cutting-edge designs integrate high-bandwidth memory controllers, dedicated AI accelerators, and sophisticated NoC interconnect systems to ensure seamless communication and scalability.
This unprecedented challenge is manageable by using advanced NoC interconnects, which serve as the backbone for efficient data transfer and communication within the chip. These on-chip networks enable seamless integration of numerous IP blocks. Moreover, high-end SoCs often rely on multiple NoCs, each tailored to specific tasks or subsystems to handle the diverse communication needs across different chip areas.
These NoCs may employ a variety of topologies, depending on the application requirements, such as rings for low-latency communication, trees for hierarchical organization, and meshes for scalability and flexibility.
To address these density and performance challenges, 3D stacking technologies are increasingly being adopted. These approaches integrate multiple layers of logic and memory vertically, enabling higher bandwidth and reduced latency compared to traditional 2D designs.
However, 3D stacking introduces additional complexity in NoC design, such as managing inter-layer communication and thermal constraints, which also require innovative interconnect solutions.
Additional challenges
The increasing sophistication of SoC designs has brought additional challenges driven by the rapid pace of growth in the market. As architectures become more elaborate, designers face mounting pressures to overcome these obstacles and adopt innovative solutions to try to keep pace with industry demands.
These challenges can be summarized as follows:
- Time-to-market pressures: Modern SoC design faces immense competition, where delays can result in significant revenue loss and diminished market share. Traditional methods like manual NoC configuration are time-intensive, often consuming weeks or months, which is unsustainable in fast-paced markets.
- Scarcity of expertise: The growing demand for specialized skills in SoC design outpaces the availability of experienced professionals. Engineering teams are often overburdened, with senior experts spending excessive time on repetitive, manual tasks rather than strategic and high-value design decisions.
- Suboptimal utilization of resources: Manual design methods often result in inefficiencies such as excessive wire lengths, increased power consumption, and physical congestion. These inefficiencies impact the overall performance and escalate both the design complexity and production costs.
- Disparate toolchains: Fragmented workflows in SoC development are a significant bottleneck, with disconnected tools used for floorplanning, connectivity and physical design. The lack of integration across these stages leads to inefficiencies, delays in achieving design closure, and difficulties in maintaining consistency throughout the design process.
Addressing these challenges requires adopting automated design methodologies, enhancing workforce expertise, and integrating toolchains to streamline workflows and reduce inefficiencies.
Designers require smarter NoC solutions
The pressure of this new wave of SoC design complexity is pushing design teams to their limits. An effective approach to managing these challenges is to divide the design into smaller, more manageable pieces by partitioning it into IP blocks.
While this method simplifies individual design tasks, it introduces a new challenge in ensuring seamless integration of these blocks to form a fully functional and optimized SoC. The integration process often reveals unexpected issues, such as mismatched interfaces, timing conflicts and resource contention, which can significantly impact performance and delay time-to-market.
The integration challenges become even more pronounced as SoC designs incorporate increasingly sophisticated components such as AI accelerators and advanced interconnect systems. For instance, the evolution of neural processor units (NPUs) and NoC technologies highlights how rapidly the complexity of SoC architectures has grown.
The first NPUs were typically implemented as arrays of multiply-accumulate (MAC) functions. By comparison, today’s NPUs are far more advanced and may be implemented as arrays of processing elements (PEs), all linked by their own mesh topology NoCs.
Similarly, NoC technology has significantly advanced. First-generation NoCs required manual layout and implementation, including the insertion of pipeline stages. Later generations of NoC technology introduced physical awareness, enabling automatic NoC generation and pipeline stage insertion.
The current generation of NoCs supports higher-end features such as soft tiling. This technology encompasses the automatic replication of processing units (PUs) such as processor clusters in high-level SoCs or PEs in NPUs. It also automatically generates the NoC and configures the network interface unit (NIU) associated with each PU with a unique address.
Features like physical awareness and NoC soft tiling dramatically increase productivity, reduce time to market, and mitigate risk. However, as design complexity continues to grow, additional advancements will be needed to address emerging challenges.
Preparing for the future of SoC design
Successfully realizing next-generation devices is getting harder, especially when it comes to integrating all the IPs into the full SoC. There is a clear and present need for the evolution of tools, including NoC technologies, to address the expanding requirements driven by market shifts such as:
- Automate repetitive and time-consuming tasks, freeing up engineering resources for innovation.
- Accelerate NoC generation without sacrificing performance, power, or quality.
- Adapt to diverse design topologies, seamlessly accommodating both hierarchical and flat NoC structures.
- Optimize across multiple metrics, including wire length, latency and congestion, to deliver high-performing designs that meet tight market windows.
- Empower engineers with user-friendly interfaces and flexible workflows, enabling incremental updates and integration into existing toolchains.
When NoC tools and technologies with these capabilities become available, SoC designers will be able to address these escalating design requirements with greater efficiency and innovation.
In short, next-generation NoC solutions must be engineered to meet today’s challenges while anticipating the accelerating demands of future SoC design.
Andy Nightingale, VP of product management and marketing at Arteris, has over 37 years of experience in the high-tech industry, including 23 years in various engineering and product management positions at Arm.
Related Content
- SoC Interconnect: Don’t DIY!
- What is the future for Network-on-Chip?
- SoC design: When is a network-on-chip (NoC) not enough
- Network-on-chip (NoC) interconnect topologies explained
- Why verification matters in network-on-chip (NoC) design
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>
The post The SoC design: What’s next for NoCs? appeared first on EDN.
