dl1.24xlarge

Analysis

The dl1.24xlarge is a machine learning asic instances instance in the dl1 series with 96 vCPUs and 768 GiB of memory on Intel Xeon Platinum 8275L (up to 3 GHz), which makes it memory-heavy at 8 GiB of memory per vCPU.

Its price is almost flat across the 2 regions that offer it ($13.11 to $13.11/hour), so latency and data residency should decide where it runs. It is not offered in Ohio, Canada Central, São Paulo, Ireland, London, Frankfurt, Mumbai, Singapore, Sydney and Tokyo, so plan around that if you need a presence there.

Looking beyond its own line, 34 of 35 current types with 96 vCPUs and similar memory are cheaper. The cheapest of those is the r5a.24xlarge on AMD EPYC 7571 at $5.42/hour.

On demand it costs $13.11/hour in US East (N. Virginia), $9,570 a month if left running, which puts it mid-table for machine learning asic instances types at the 65th percentile per vCPU ($0.1366 per vCPU-hour) and the 35th percentile per GiB ($0.0171 per GiB-hour).

It has local instance storage (4 x 1000 GB NVMe SSD), which is fast but lost when the instance stops. Network: 400 Gigabit.

Reserving it for a year with no upfront payment cuts the Linux rate to $8.26/hour, which beats on demand once it runs more than 63% of the time; three years brings it to $5.66/hour, 57% below on demand.