Keyboard shortcuts

/ or ⌘/Ctrl K
Find a note
j / k
Next / previous section or linked note
h / l
Collapse or go to parent / expand or enter
e or Alt-click
Read a linked note here
o
Open focused note on its own
g g / G
First / last section or linked note
g h / g a
Home / all notes
g b / g t
Backlinks / table of contents
t
Cycle System, Light, Dark
? / Esc
Show / close this reference

Search: ↑/↓ or Ctrl N/P, Enter to open. Shortcuts pause while typing.

sectionPaging [812ab3d6]

Part of linux
  • Page sizes traditionally 4kb, but really if you're a big boy we're all huge pages now
  • There's a page table, and we represent the address space of the program and how it's mapped into the actual ram

  • One of the bigger issues with paging is that storing the page table and all it's constitutant information can get quite large. So we avoid mapping all the pages

    • When a process generates a virtual address, the OS and the hardware have to make it a meaningful real address
    • note that if we store page tables in memory, hitting RAM would be prohibitively slow, so we do a TLB
  • page table entries change logical memory spaces into physical spaces, how they ordered, how they used, varies widely
  • you don't have to map all the pages, but if they're not in ram, where are they?
  • kernel allocates the pages, and mapped when a program is loaded at execute time

  • shared libraries (shared among various processes, chances are .so pages are already in RAM, so they map to another process address space)

    • although now static binaries might be good enough since we have tons of ram now

  • context switching doesn't involve memory changes

    • means switching to different address spaces and different page tables
    • address spaces can sit all in memory with enough ram, just change the register values, no freeing memory
    • pages are copied out to swap (AND HIT DISK)
    • reason why it's called "swap", to swap processes between context switches
    • now you just move the pages out as you need to

  • typically a disk partition or a file, and we can have multiple swap spaces

    • mkswap(8) and swapon(8)

    • swapon -s in terms of unix blocks

      • unix blocks are 512 bytes, each block is half a K, although now there's different

    2024-02-18_18-46-21_screenshot.png

  • Possible to have memory on the swap space and in memory!

  • "dirty page" - means that one copy that has been created is not the same as a the one in swap anymore

    • dirty pages need to reswap out
  • A "page fault" happens when you hit something that is not in RAM, which triggers a copy back from swap
  • happens on demand, which means paging in and out requires a very long wait
  • page ins are synchronous! the worst case

  • Linux KPTI:

    2025-01-05_20-34-20_screenshot.png

    • separate page table, isolates user space and kernel space (against Meltdown), 5-30% performance degredation, causes partial TLB flushes
    • pmap gets you memory mappings

1. ASLR

  • Where is it controlled? /proc/sys/kernel/randomize_va_space

2. 48 bit addressing

3. pgd, pud, pmd, etc

2025-12-08_20-16-57_screenshot.png

  • Different layers of pgd

  • Why do we even have multilevel paging?

    • Because we want to reduce the amount of RAM. If we had a single level, the page table would have to be massive, it would have to allocate the whole table.
    • Multilevel is a directory of pages
  • lwn.net/Articles/717293/

4. Huge TLB (hugetlbfs)

  • www.youtube.com/watch?v=n67gCNiKVcw
  • hugetlbfs has been used in the past to reduce the cost of memory translation like any hugetlb, but there's some issues
  • But this pins memory, we can't swap huge pages to disk, and no THP stalls.
  • Transparent Huge Pages can try to act like huge pages, but the merging mechanism can cause CPU spikes and jitter. Same as compaction in LSM trees
  • Get default huge page size in /proc/meminfo
  • You can populate huge tlb on kernel boot, or at runtime
  • hugepage_cma -> contigious memory allocator
  • You can do it at boot time or at runtime, but doing at boot is a lot easier because of fragmentation
  • You can mount a hugetlbfs filesystem, and all files in that filesystem are backed by huge pages
  • You can't swap or reclaim huge pages, so you just lose them, end up with a SIGBUS
  • We can share the PMD entries in the table for hugetlbfs

  • What kind of applications benefit from huge pages specifically?

    • Databases engines

      • Original users, they have a shared memory arena that each query process checks, so less memory lookups

    • Java/JVM

      • JVM allocates heap as one continuous block. This makes the GC scans faster by having to scan less on memory translation
      • -XX:+UseLargePages

    • DPDK

      • Bypass kernel stack, you need huge pages just for dealing with packet drops

    • AI/ML Training

      • pin_memory=True in pytorch speeds up memory usage because you need to shuttle around a lot of data

  • What about not huge pages?

    • redis

      • Uses fork() aggressively
      • Relies on copy on write, but child procesess have to copy the entire page

    • webservers

      • Not a lot of gain because requests are short lived and memory is already fragmented

5. Arm Paging: wiki.osdev.org/ARM_Paging