Oracle Exadata X11M: Hardware Architecture Overview
The core architecture of Oracle Exadata is based on a scale-out design featuring database servers, intelligent storage servers, and a high-speed, low-latency network fabric connecting these components. Exadata X11M combines this architecture with the latest hardware technologies and optimized software, creating a unique platform for Oracle Database workloads. The co-engineered nature of hardware and software aims to maximize performance at every layer of the system.
Database Server (Standard X11M)
Exadata X11M database servers are designed for compute-intensive database operations. Key hardware specifications include:
- CPU: Equipped with two AMD EPYC™ 9J25 processors, each featuring 96 cores per socket (2.6 GHz base, up to 4.5 GHz boost), totaling 192 physical cores per server. These cores are stated to be up to 25% faster than their X10M counterparts.
- RAM: Utilizes high-performance 6400MT/s DDR5 DIMMs. Memory capacity options per server are 512 GB (16x32GB), 1.5 TB (24x64GB), 2.25 TB (24x96GB), or 3 TB (24x128GB). This provides up to 33% more DRAM bandwidth compared to X10M.
- System Storage: Standard configuration includes 2 x 3.84 TB NVMe devices, expandable to 4. Typically used for the OS and system software.
- RDMA Network Fabric: Features one dual-port ConnectX-7 (CX7) RDMA Network Fabric adapter (PCIe 5.0) providing 2 x 100 Gb/s RoCE (RDMA over Converged Ethernet) ports in an active-active configuration for a total bandwidth of 200 Gb/s.
- Client/Management Network: Includes dedicated 1GbE Base-T ports for Admin and ILOM. Two dual-port 10/25 GbE adapters (CX6-LX) are factory-installed for client connectivity. Optional adapters include additional 10/25 GbE, 100 GbE (CX6-DX), or Quad 10GBASE-T.
Storage Servers (HC and EF)
Exadata storage servers not only store data but also intelligently offload significant portions of SQL query processing from the database servers, reducing CPU load and network traffic. The X11M generation offers High Capacity (HC) and Extreme Flash (EF) storage server types:
- CPU: Both HC and EF servers feature two AMD EPYC™ 9J15 processors (32 cores per socket, 2.95 GHz base, up to 4.4 GHz boost), providing 64 cores per server dedicated to SQL offload and system operations. These are up to 11% faster than X10M storage server CPUs.
- RAM: Each server contains 1.5 TB of 6400MT/s DDR5 DRAM, offering 33% more bandwidth than X10M.
- Exadata RDMA Memory (XRMEM): A significant portion of the RAM, 1.25 TB, is configured as XRMEM, an ultra-low latency cache tier. Accessible directly via RDMA, XRMEM sits between the database buffer cache and Flash Cache, enabling OLTP read latencies as low as 14µs. RDMA bypasses much of the OS and network stack overhead, accessing DRAM (XRMEM) which is inherently faster than flash, to achieve this low latency.
- Exadata Smart Flash Cache: Utilizes performance-optimized PCIe 5.0 NVMe flash cards (F680 v2).
- HC Server: 4 x 6.8 TB cards (Total 27.2 TB cache).
- EF Server: 4 x 6.8 TB cards (Total 27.2 TB cache).
- This new generation flash provides up to 2.2X faster analytical I/O (scan throughput) compared to X10M. SQL Scan throughput from flash reaches 100 GB/s per server.
- Persistent Storage:
- HC Server: 12 x 22 TB 7,200 RPM SAS HDDs (Total 264 TB raw capacity).
- EF Server (All-Flash): 4 x 30.72 TB capacity-optimized NVMe PCIe Flash drives (Total 122.88 TB raw capacity).
- System Storage: 2 x 480 GB NVMe devices (likely for OS/Exadata software).
- RDMA Network Fabric: Same as database server: 1 x dual-port CX7 adapter (PCIe 5.0), 2×100 Gb/s active-active RoCE (Total 200 Gb/s).
The synergy between faster AMD EPYC cores, higher-bandwidth DDR5 RAM, significantly faster PCIe 5.0 flash scans, and the large XRMEM cache accessed via the 200 Gb/s RoCE network forms the foundation for X11M’s overall workload acceleration. Performance gains stem from the interaction of these components (e.g., faster CPUs fed by faster memory/flash over a faster network) rather than any single element.
Table 1: Exadata X11M Server Component Specifications
| Component | Database Server (X11M) | Storage Server (HC) | Storage Server (EF) |
|---|---|---|---|
| CPU | 2 x AMD EPYC™ 9J25 (96 Cores/socket, 192 Total) | 2 x AMD EPYC™ 9J15 (32 Cores/socket, 64 Total) | 2 x AMD EPYC™ 9J15 (32 Cores/socket, 64 Total) |
| RAM (Total Size) | 512GB / 1.5TB / 2.25TB / 3TB | 1.5 TB | 1.5 TB |
| RAM (Type/Speed) | DDR5 / 6400MT/s | DDR5 / 6400MT/s | DDR5 / 6400MT/s |
| XRMEM Size | N/A | 1.25 TB | 1.25 TB |
| Flash Cache (Type) | N/A | Perf. Opt. F680 v2 NVMe PCIe 5.0 | Perf. Opt. F680 v2 NVMe PCIe 5.0 |
| Flash Cache (Size) | N/A | 4 x 6.8 TB (Total 27.2 TB) | 4 x 6.8 TB (Total 27.2 TB) |
| Persistent Storage (Type) | System NVMe (2×3.84TB, exp. to 4) | SAS HDD (7200 RPM) | Cap. Opt. NVMe PCIe Flash |
| Persistent Storage (Size) | – | 12 x 22 TB (Total 264 TB Raw) | 4 x 30.72 TB (Total 122.88 TB Raw) |
| RDMA Network | 1x Dual Port CX7 (PCIe 5.0), 2x100Gb/s Active-Active RoCE | 1x Dual Port CX7 (PCIe 5.0), 2x100Gb/s Active-Active RoCE | 1x Dual Port CX7 (PCIe 5.0), 2x100Gb/s Active-Active RoCE |
Introducing the Exadata X11M-Z Variant
Alongside the standard high-performance configurations, the Exadata X11M family introduces a more economical entry-level option designated “X11M-Z”. This variant is designed for customers with smaller workloads or those seeking access to core Exadata capabilities at a lower cost point. The X11M-Z replaces the fixed “Eighth Rack” concept of previous generations, offering a more flexible starting configuration. Customers can start with X11M-Z servers and scale out by adding more Z servers or standard X11M servers as needs grow. This approach potentially broadens Exadata’s market reach by attracting customers with smaller initial needs or budgets who might otherwise opt for less optimized platforms or cloud services lacking Exadata features.
X11M-Z Database Server Specifications
- CPU: Utilizes a single-socket configuration with 1 x 32-core AMD EPYC™ 9J15 processor (2.95 GHz base, up to 4.4 GHz boost).
- RAM: Uses 6400MT/s DDR5 DIMMs. Memory options are 768 GB (12x64GB) or 1.125 TB (12x96GB).
- System Storage: Standard 2 x 3.84 TB NVMe devices, expandable to 4.
- RDMA Network Fabric: Same as standard X11M DB server: 1 x dual-port CX7 adapter (PCIe 5.0), 2×100 Gb/s active-active RoCE (Total 200 Gb/s).
- Client/Management Network: Similar options to standard X11M DB server, but with only one optional field-installable adapter slot.
X11M-Z High Capacity (HC-Z) Storage Server Specifications
- CPU: Single-socket configuration with 1 x 32-core AMD EPYC™ 9J15 processor.
- RAM: 768 GB 6400MT/s DDR5 DRAM.
- XRMEM: 576 GB of the RAM is allocated as XRMEM.
- Flash Cache: Two 6.8 TB performance-optimized F680 v2 NVMe PCIe 5.0 flash cards.
- Persistent Storage: Six 22 TB 7,200 RPM SAS HDDs.
- System Storage: 2 x 480 GB NVMe devices.
- RDMA Network Fabric: Same as standard HC/EF storage servers: 1 x dual-port CX7 adapter (PCIe 5.0), 2×100 Gb/s active-active RoCE (Total 200 Gb/s).
Positioning and Use Cases
The X11M-Z variant is ideal for smaller databases, development/test environments, or departmental applications that require core Exadata software features (Smart Scan, Storage Indexes, HCC, IORM) and performance characteristics but do not need the full scale of standard X11M. It offers a lower entry cost and a reduced power consumption footprint. The single-socket design inherently consumes less power than dual-socket standard servers, aligning with the overall theme of improved energy efficiency and sustainability in X11M. Offering a lower-power hardware option alongside software-based power management strengthens Exadata’s efficiency narrative.
Table 2: Exadata X11M vs. X11M-Z Key Hardware Differences
| Component | Standard X11M Server | X11M-Z Server |
|---|---|---|
| Server Type | Database / Storage (HC/EF) | Database / Storage (HC-Z) |
| CPU Sockets | 2 | 1 |
| CPU Cores/Server | DB: 192 / Storage: 64 | DB: 32 / Storage: 32 |
| Max RAM/Server | DB: 3 TB / Storage: 1.5 TB | DB: 1.125 TB / Storage: 768 GB |
| XRMEM Size (Storage) | 1.25 TB | 576 GB |
| Flash Cache (Storage) | 4 x 6.8 TB Cards (HC/EF) | 2 x 6.8 TB Cards (HC-Z) |
| Disk Drives (HC) | 12 x 22 TB | 6 x 22 TB (HC-Z) |
Generational Performance Leap: Exadata X11M vs. X10M
Exadata X11M delivers significant performance increases across all major workload types compared to the previous X10M generation. These gains stem from the combination of updated hardware components (CPU, memory, flash, network) and continuous software optimizations. Oracle offering these improvements at the same starting price as X10M enhances the platform’s value proposition.
AI Vector Search Acceleration
With the rise of AI-powered applications, AI Vector Search for semantic similarity search, RAG applications, recommendation systems, and anomaly detection has become critical. Exadata X11M is explicitly optimized for these workloads.
Exadata’s unique AI Smart Scan feature plays a central role by offloading vector distance calculations and top-K filtering to storage servers. Data is processed in XRMEM or Flash Cache on the storage servers, minimizing network traffic and database server load.
X11M provides the following concrete gains over X10M for AI Vector Search:
- Persistent Vector Index (IVF) Searches: Up to 55% faster due to hardware acceleration and transparent storage offload.
- In-Memory Vector Index (HNSW) Queries: Up to 43% faster, benefiting from faster database server CPUs and memory.
- Software Optimizations (All Exadata Platforms):
- 4.7X more data filtering on storage servers (due to improved top-K efficiency).
- Up to 32X faster queries when searching BINARY vector dimensions. This offers significant speedup with minimal accuracy impact for some use cases.
- Offload of vector distance projection to storage servers.
OLTP Performance Enhancements
Low latency and high IOPS are crucial for financial transactions, e-commerce, and other critical OLTP systems. Exadata X11M delivers significant improvements:
- IOPS: Delivers up to 25.2 Million 8K SQL read IOPS in a single rack. Competitive claims mention significantly higher IOPS compared to rivals like Pure Storage (33x) and Dell PowerMax (3.3x).
- Latency: SQL 8K read latency is reduced to as low as 14 microseconds (14μs) thanks to XRMEM and RDMA. This is a 21% improvement over X10M’s 17µs. Cloud comparisons claim up to 70x lower latency than AWS RDS and Azure SQL.
- Throughput: Up to 25% faster serial transaction processing and up to 25% higher concurrent transaction throughput compared to X10M, driven by faster cores.
- Flash Performance: Single block reads from flash are up to 43% faster.
Analytics Workload Acceleration
Data warehouses, reporting, and large-scale analytics demand high scan throughput. Exadata X11M shows significant progress here:
- Scan Throughput:
- Analytical I/O (scan throughput) on storage servers is up to 2.2X faster than X10M. This is enabled by faster PCIe 5.0 flash and potentially the ability to scan from both flash and XRMEM concurrently.
- Flash scan throughput reaches 100 GB/s per server.
- Scanning columnar data cached in XRMEM reaches 500 GB/s per server.
- Total scan throughput per rack can reach up to 8.5 TB/s (likely from XRMEM). This is a substantial increase from the 1 TB/s per rack claimed for X10M.
- Query Processing: Up to 25% faster analytics query processing compared to X10M. This results from faster database server cores and faster storage offload processing.
- Database In-Memory: Database In-Memory scans increase up to 500 GB/s. Exadata extends In-Memory columnar formats into Flash Cache and XRMEM, leveraging ultra-fast SIMD Vector instructions.
This balanced improvement across workloads reinforces Exadata’s suitability for diverse and consolidated environments. The significant gains allow more demanding workloads to coexist efficiently on the same platform. While hardware upgrades provide the raw potential, software optimizations like AI Smart Scan enhancements, RDMA protocols, and Exadata System Software unlock this potential. Some software optimizations, like the 32x faster binary vector queries, are available across Exadata platforms (with appropriate software), demonstrating software value independent of hardware, but achieving peak potential on the latest hardware like X11M.
Key Software and Efficiency Improvements
Beyond raw performance, Exadata X11M introduces significant software and hardware features focused on improving operational efficiency and resource utilization, helping to lower Total Cost of Ownership (TCO) and address modern data center concerns like sustainability and management overhead.
Advanced Power Management
Introduced with Exadata System Software 25.1 and specifically designed for X11M database servers, these capabilities help organizations meet energy efficiency goals and reduce operational costs. Key features include:
- Core Disablement: When the active core count (
pendingCoreCount) is set to 128 or lower, the system automatically powers off 64 unused CPU cores (32 per socket), saving approximately 80 Watts per server without impacting performance for the licensed cores. Cores can be re-enabled if needed. - Power Capping: Allows setting an overall power consumption target for the database server, useful for regulatory compliance or cooling management. Performance scales linearly with the power cap (e.g., a 10% power reduction results in roughly a 10% peak processing capacity reduction).
- Low Power Mode: Enables scheduling automatic transitions to a low power mode during anticipated low-usage periods (e.g., nights, weekends). The system automatically exits low power mode if demand unexpectedly increases (based on CPU, I/O, or network thresholds) to maintain performance.
Combined with the ability to consolidate more workloads onto fewer systems due to X11M’s higher performance, these power management features can lead to significant savings in infrastructure, power, cooling, and data center space costs.
Storage Efficiency
- Exascale Free Space Management (Exadata SW 25.1): This enhancement significantly reduces the amount of free space Exascale storage pools require compared to traditional ASM disk groups to successfully complete an automatic data rebalance after a storage device failure. For example, with 9+ HC servers, Exascale requires only 3% free space versus 9% for ASM. This increases usable storage capacity. Such improvements indicate the maturation of the Exascale architecture beyond basic pooling.
- Hybrid Columnar Compression (HCC): A standard Exadata feature that significantly compresses data (often 5x-20x), especially for analytics, saving storage costs and improving performance by reducing I/O.
- Exascale Thin Cloning: Redirect-on-write technology enables space-efficient clones, particularly useful for dev/test environments.
Other Notable Software Features (Exadata SW 25.1)
Exadata System Software 25.1 introduces other improvements relevant to the X11M platform:
- Automatic Tuning of ASM Rebalance: Dynamically adjusts the
asm_power_limitbased on available I/O bandwidth and client database workload presence. It speeds up rebalance when resources are free and slows it down to prioritize user workloads, minimizing performance impact and reducing manual tuning needs. This contrasts with the traditional static power limit approach. - Simpler Package Management: Streamlines management of additional non-Exadata software packages during database server updates (that don’t change the major OS version), reducing maintenance window duration.
- Exascale Volume Cloning: Ability to create clones directly from existing Exascale volumes.
- Secure Fabric Default: Secure internal communication layer is now recommended and enabled by default.
- Cache Observability Enhancements: Improved monitoring of Exadata cache performance via
ecstatutility enhancements. - Faster Cisco Switch Upgrades: Reduces downtime associated with network switch software updates.
These efficiency-focused features demonstrate that X11M offers a holistic approach, addressing not just raw speed but also operational costs, management complexity, and environmental impact.
Comparative Summary: Exadata X11M vs. X10M
Exadata X11M represents a significant step up from its predecessor, X10M, offering substantial hardware and software enhancements that translate into tangible performance and efficiency gains. The table below summarizes key technical and performance differences:
Table 3: Feature and Performance Comparison (Exadata X11M vs. X10M)
| Feature/Metric | Exadata X10M | Exadata X11M | Improvement |
|---|---|---|---|
| Hardware | |||
| DB Server CPU | 2x AMD EPYC™ 9J14 (96 Cores/socket) | 2x AMD EPYC™ 9J25 (96 Cores/socket) | Up to 25% faster core performance |
| Storage Server CPU | 2x AMD EPYC™ (32 Cores/socket) | 2x AMD EPYC™ 9J15 (32 Cores/socket) | Up to 11% faster core performance |
| Memory Type/Speed | DDR5 / 4800MT/s | DDR5 / 6400MT/s | 33% more bandwidth |
| Flash Technology | PCIe 4.0 NVMe | PCIe 5.0 NVMe | Faster (Up to 2.2x for Analytics I/O) |
| Network Fabric Speed | 2x100Gb/s RoCE (Active-Active) | 2x100Gb/s RoCE (Active-Active) | Same nominal speed |
| XRMEM Latency | < 17µs | < 14µs | Up to 21% lower |
| AI Vector Search | |||
| IVF Search Speedup | Baseline | Up to 55% faster | Up to 55% |
| HNSW Search Speedup | Baseline | Up to 43% faster | Up to 43% |
| Binary Vector Query Speedup | Baseline (Software dependent) | Up to 32x faster (Software Opt.) | Up to 32x (Software Opt.) |
| OLTP | |||
| Max Read IOPS (per rack) | 25.2 Million | 25.2 Million | Equal (Likely network/protocol limited) |
| SQL Read Latency | < 17µs | < 14µs | Up to 21% lower |
| Serial Transaction Speedup | Baseline | Up to 25% faster | Up to 25% |
| Concurrent Throughput Increase | Baseline | Up to 25% more | Up to 25% |
| Analytics | |||
| Analytics I/O Speedup (Storage) | Baseline | Up to 2.2x faster | Up to 2.2x |
| Analytics Query Processing Speedup | Baseline | Up to 25% faster | Up to 25% |
| Max In-Memory Scan Speed (Server) | ~227 GB/s (Implied/Est. for X10M) | 500 GB/s (from XRMEM) | >2x increase (with XRMEM scan) |
| Efficiency | |||
| Advanced Power Management Features | No | Yes (Core disable, Capping, Low Power Mode) | New |
| X11M-Z Option | No (Eighth Rack existed) | Yes | New (More flexible entry-level) |
]
Conclusion
Oracle Exadata X11M is a powerful engineered system representing a significant advancement over the previous X10M generation in performance, efficiency, and flexibility. The integration of the latest AMD EPYC processors, faster DDR5 memory, PCIe 5.0 flash, and Exadata RDMA Memory (XRMEM) provides a substantial hardware uplift.
These hardware improvements, combined with intelligent software optimizations like AI Smart Scan and features introduced in Exadata System Software 25.1 (Advanced Power Management, Automatic ASM Rebalance Tuning, enhanced Exascale storage management), further amplify X11M’s capabilities. The platform delivers demonstrable performance gains across critical workloads, including AI Vector Search (up to 55% faster IVF, 32x faster binary queries), OLTP (up to 25% higher throughput, latency down to 14µs), and Analytics (2.2x faster storage I/O, up to 500 GB/s In-Memory scans).
Crucially, these advancements are offered at the same price point as the previous generation, making X11M a compelling value proposition. Enhanced power management and consolidation potential contribute to lower TCO and improved sustainability, while the introduction of the X11M-Z variant makes the platform accessible to a broader range of customers.
With deployment flexibility across on-premises, OCI, Cloud@Customer, and major multicloud environments (Azure, AWS, Google Cloud), Exadata X11M allows organizations to run their critical Oracle Database workloads wherever needed without application changes.
In summary, Oracle Exadata X11M stands as one of the most advanced and strategic platforms available for organizations seeking peak performance, scalability, and efficiency for their Oracle Database workloads. Its strong emphasis on AI capabilities signals its readiness for future enterprise computing trends.