How AI Is Redefining the Role of Memory: From Component Cost to Strategic AI Infrastructure

下午 2:05 - 下午 2:25

AI is redefining the role of memory in computing systems. Once treated mainly as a cost and power component, memory has become a strategic driver of AI performance, scalability, and competitiveness. As AI shifts from large-scale training to inference at massive scale, memory capacity, bandwidth, and efficiency increasingly determine key outcomes such as tokens per watt and tokens per dollar.

This keynote explores how memory is evolving beyond data storage to hold intelligence, knowledge, context, agent states, and collaboration history. Long-context inference, RAG, reasoning systems, and multi-agent AI are rapidly increasing memory demand, while KV cache and agent workflows create new bottlenecks tied to service complexity.

The talk also examines innovations aimed at reducing data movement, including compute-near-memory, virtual HBM pooling, CXL-based fabric-attached memory, and long-term AI memory orchestration. Finally, it argues that memory providers must move beyond component supply to become system co-design partners for memory-centric AI architectures.

Key Technologies Covered

Long-Context AI Inference and KV Cache Scaling
Compute-Near-Memory for Energy-Efficient AI Systems
Shared Memory Architecture: CXL-Based Memory Disaggregation
AI Memory Hierarchy and Long-Term Memory Orchestration

Featured Speakers

Dr. Hoshik Kim

Senior Vice President, SK hynix Inc.

Hoshik Kim is Senior Vice President and Fellow at Memory Systems Research, SK hynix Inc., where he leads research and pathfinding in memory systems architecture and software solution for data centers at scale as well as edge devices. His current research interests focus on next-generation memory systems for AI and HPC, which include memory expansion, tiering, pooling and sharing as well as computational memory/storage solutions such as PIM/PNM, custom HBM, etc. He is also interested in advanced packaging and emerging interconnect technologies for better compute and memory integration. Prior to joining SK hynix, he worked for Intel and LG Electronics, where he gained broad experiences in architecture, design, verification and electronic design automation (EDA) for microprocessors, SoC’s and IP’s. Hoshik received B.S. from Yonsei University, Seoul, Korea and M.S. and Ph.D. from University of Southern California, all in Electrical Engineering. He is also serving on the Board of Directors for Semiconductor Research Corporation (SRC). Contact him at [email protected].