INT410 - Windows Memory Management Internals

This seminar provides a intensive, detailed study of the memory manager of current Windows operating systems.
Level: Advanced
Audience: Systems software developers; device driver developers; hardware OEMs.
Description:

This seminar describes both the operational principles and implementation details of the executive memory manager in Windows. All significant data structures are described. Typical call trees for common paths through the memory manager code are presented with the aid of the Windows debugger.

Particular attention is given to changes and improvements made in the most recent versions (Windows Vista, Windows Server 2008, Windows 7, and Windows Server 2008 R2).
Topics:
  • Review of essential principles
    • Processes and address spaces
    • VMMap SysInternals tool
    • Virtual address translation
    • Page faults
    • Setting up the Windows debugger
  • Page fault details
    • Page fault walkthrough
    • Page table entry formats and semantics
    • Other memory access exceptions: Access violations, no-execute
  • Physical memory management
    • Working set list structure
    • Page replacement algorithm
    • Modified and standby page lists
    • Page writer threads
    • Memory priority and standby page list
    • Non-Uniform Memory Access (NUMA) platforms
    • Free and zero page list; zero page thread
    • Balance set manager
    • RamMap SysInternals tool
    • Memory manager synchronization methods
  • Virtual address space definition and backing stores
    • Free, reserved, and committed address space
    • Page files
    • Virtual address descriptors
    • Shared virtual address space; mapped files; sections (file mapping objects)
    • Copy-on-write pages
    • Mapping exe's, dll's, and other code files
    • Prototype Page Table Entries and Control Areas
    • Address Space Layout Randomization
  • User mode issues
    • Image activation
    • Process creation and rundown
    • Side-by-side DLLs
    • User mode heaps
  • File system cache
    • Basics - mapping views
    • File open options
    • Interaction with file system drivers
    • Use of non-virtually-mapped physical pages
    • File placement optimization and prefetch
    • SuperFetch
    • ReadyBoost
  • Platform features and Windows
    • CPU cache, MESI protocols, etc.
    • Address translation buffer
    • LOCKed memory operations
    • x86/x64 MMU capabilities
    • Modern platform architectures (QPI / HyperTransport)
    • Non Uniform Memory Access
    • Code and data in ROM: The ROM page list
    • Physical memory addressing and licensing limits
  • Kernel mode issues
    • Kernel stack basics
    • Kernel stack guard pages, growth, stack switching
    • Dynamic kernel address space allocation
    • System working sets
    • System page table entries
    • Memory manager and hypervisor support
    • The .PhysicalMemory device
    • Non Uniform Memory Access
  • Driver issues
    • The PHYSICAL_ADDRESS type
    • DMA under WDM
    • DMA under KMDF
    • Mapping kernel memory to kernel space
    • Mapping kernel memory to user space
  • The very last word on...
    • Physical Address Extension and the "4 GB barrier"
    • The "3 GB barrier"
    • Memory counters and terminology
    • Pagefile size and placement
    • Memory manager fads and fallacies in the name of performance
Prerequisites: INT201, Windows Internals, or INT250, Windows Internals Workshop, or equivalent experience with knowledge of Windows internals.
Operating systems supported: This seminar primarily addresses Windows 7 through Windows 10 and Windows Server 2012 R2. Most of the material is applicable to earlier versions of Windows. Earlier versions can be specifically addressed upon request.
Durations and formats: 4 days with labs
Labs: The lab exercises in this seminar involve use of various Windows and SysInternals tools to explore the operating system and confirm the behaviors described. We will use the Windows debugger to explore various code paths through the memory manager code, in particular the pager. Due to the large amount of detail, this seminar is not offered without labs. Labs are essential to understand and retain the information presented.