Research Note: Arm announces new third-generation Neoverse designs

On 21^st February 2024, Arm Holdings PLC announced two new additions to the Neoverse family of processor core designs. The V3 and N3 extend the performance capabilities of Arm’s flagship data centre platform. We take a look at what these new solutions offer and how they improve the capabilities of current-generation solutions already in the market.

Background

Arm Holdings PLC (generally referred to as Arm) develops processor IP widely used across the industry in a broad range of use cases that includes smartphones, mobile devices, and edge computing. In October 2018, the company announced the Neoverse platform, which focused on data centre and enterprise computing (server-based deployments).

The first architecture to be announced was codenamed Ares, which launched in February 2019, based on a 7nm process. The initial roadmap also referenced Zeus, using a 7nm+ process, which was officially announced in September 2020 and Poseidon, which we will discuss in a moment. The codenamed architectures have developed into three product families as follows.

V-Series – the highest performing cores focused on optimising TCO.
N-Series – efficient performance per watt, optimising power/performance.
E-Series – designed for throughput efficiency (edge use cases).

Ares became the first generation Neoverse N1, while Zeus was launched as the Neoverse V1. Arm extended these architectures with the N2 release in September 2020 (coinciding with V1), while Neoverse V2 was announced in September 2022.

We briefly discussed the Neoverse roadmap in a blog post covering the announcement of Graviton4 back in December 2023. As the Arm published roadmap indicates, Poseidon is the third generation of the Neoverse V-series architecture, which was announced in February 2024.

Version 3

Both V-Series and N-Series Neoverse have been extended with the announcement of Neoverse V3 (codenamed Poseidon) and Neoverse N3. The V3 design is based on the Armv9-A (v9.2) instruction set architecture, with SVE2 (scalable vector extensions), up to 3MB L2 ECC cache and 64KB of both instruction and data L1 cache. The N3 specifications are slightly lower with up to 2MB L2 cache and either 32KB or 64KB each of instruction/data L1 caches.

In both cases, the version 3 architectures are now offered within the Neoverse Compute Subsystem (CSS), a set of pre-integrated and validated configurations that save customers time (and cost) in designing Arm cores into SoCs (Systems on Chip). This offering was announced in August 2023.

Figure 2 – Performance improvements for gen 3

The configurations for Neoverse CSS V3 are up to 128 cores in multiple options in either single or two-socket configurations, 12 channels of DDR5 or LPDDR5 memory, 64 lanes of PCIe Gen 5 or CXL 3.0 I/O and UCIe 1.1 die-to-die links.

The Neoverse N3 delivers 32 cores per die, with 4 channels of DDR5/LPDDR5 memory and 32 lanes of PCIe Gen5 or CXL 3.0 I/O. Neoverse CSS N3 designs support as few as 8 cores or up to 192 cores for hyperscale designs.

Improvements

Of course, none of the specifications matter without a view of performance. As Figure 2 shows, Arm is expecting a typical average 15% improvement for Neoverse V3 workloads, with 84% improvement in AI/analytics. Similarly, N3 offers from 9-30% improvement, depending on workload and 196% improvement for AI/analytics. Arm also highlights a 20% improvement in performance per watt for N3.

These data aren’t validated and are Arm simulated figures, but should reflect a significant step up, especially for AI workloads. We look forward to seeing how these new designs play out when introduced by the cloud hyper-scalers.

Figure 3 – Comparison performance improvements

The Architect’s View®

As we highlighted in our post from December 2023, Neoverse V1 was used for AWS Graviton 3, while Graviton4 uses Neoverse V2. As Neoverse N2 is already available, we suggested a possible “Graviton4 Lite” configuration. It’s likely that we should expect V3 to become Graviton5, with an announcement probably at Reinvent in November 2024.

Similarly, Microsoft’s Cobalt 100 is understood to be based on Neoverse N2, providing room for a quick expansion into N3 and V-series designs. As a result, we should expect Microsoft to iterate quickly once Cobalt 100 goes into production. See our coverage of this Microsoft announcement here.

In September 2023, we covered the Google announcement of new AmpereOne processors into GCP, based on a custom Armv8.6+ ISA design. We can see that Arm is quickly gaining traction in the data centre, with a quickly increasing depth and breadth of offerings. Currently, this seems to be mainly in the public cloud, rather than on-premises data centres.

The third-generation announcements are interesting because they start to establish a cadence on the time period between which we can expect new solutions. With three product lines (of which two look best suited to the cloud), Arm can offer both performance and power-optimised options.

We should also remember that the Neoverse platform is used within other solutions, such as NVIDIA Grace (Neoverse V2 with 144 cores) and SmartNICs. Arm is expanding into the data centre, addressing price/performance/power efficiency concerns and as an adjunct for high-performance GPUs. This currently appears to be the route for on-premises solutions, although we may see this change over time.