Scheduler and concurrency

Part I · 31 threads in 2 era slices, from Go 1.0 to Go 1.28. Each thread gives its design, then a per-release timeline of what landed (Done) and what was still open. Release labels in a timeline link to that release's page.

● = the thread had substantive commits in that cycle; hover for the stage summary, click to jump to it in the release page.

Thread map, Go 1.0 – 1.17

Thread map, Go 1.18 – 1.28

Scheduler and concurrency runtime (Go 1.0–1.17)

Reader: Claude Sonnet. Cites 1,127 of 1,521 commits in this slice; every non-noise commit is cited, plus 15 larger commits the noise filter had set aside, which a separate gap reader described. Read the opening lines of the design-heavy messages and about 18 full messages. The other commits were placed from their subject lines, and some later-release bullets are short.

The G-M-P scheduler: from global-lock M:N to per-P run queues, spinning Ms and work stealing

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (findrunnable had just been split into helpers for 1.18) → continues as Scheduler core: spinning Ms, work conservation, stealing, STW interaction (1.18+)

Paths not taken: NUMA-aware scheduler (never landed)

Design. The 2008 scheduler (e7d549fbd7, “more fifo scheduling algorithm, newproc will reuse dead procs”) ran all goroutines (G) on a single global run queue protected by one sched lock. Multithreading arrived with d28acc42ec (“first cut at multithreading, works on Linux”): worker threads (M) took Gs from the global queue and an mcpu count limited how many ran Go code at once. $GOMAXPROCS was redefined as “number of CPUs to use, not number of threads” (efc86a74e4) and exposed as runtime.GOMAXPROCS (7955490de2). Every transition (ready, block, syscall) went through the global lock; by 2011 the work was shaving that lock off the syscall path (entersyscall/exitsyscall via atomics, 025abd530e, then xadd instead of cas, ba134539c5).

Go 1.1 replaced this with Dmitry Vyukov’s design: the processor P owns a local run queue, a memory cache and a cache of dead Gs; an M must hold a P to run Go code, and GOMAXPROCS is now the number of Ps. The preparation was done in steps: f20fd87384 introduced park() to atomically unlock a mutex and park a goroutine (removing the racy readyonstop flag that needed the global mutex), e25f19a638 added entersyscallblock, 353ce60f6e local work queues, 6cdfb00f4e per-P dead-G cache and the global runnable queue, and 779c45a507 (“improved scheduler”, a ~1,800 line rewrite of proc.c) made the switch: per-P runnable queues, per-P mcache and G cache, faster non-blocking syscall enter/exit and “more conservative worker thread blocking/unblocking” (idle Ms park, only a bounded number spin looking for work).

From there the work is steady refinement of fairness, wakeup policy and stealing. The invariants that matter: a P with runnable work must have an M looking at it (spinning-M accounting, nmspinning); ready() only wakes another M when no M is already spinning; a P entering syscall is retaken by sysmon after 20us/one sysmon tick and handed off with handoffp; idle Ps live on sched.pidle; runnable Gs overflow from a P’s 256-slot ring to the global queue.

Timeline.

• Go 1.0 — Done: the pre-1.1 scheduler.
  • 2008: the first FIFO scheduler e7d549fbd7; first multithreading on Linux d28acc42ec; Mac kernel semaphores used only on real contention 5ff12f871f and the per-G Note removed a61bb95497; GOMAXPROCS redefined efc86a74e4, runtime.GOMAXPROCS added 7955490de2; LockOSThread/UnlockOSThread for the debugger 218c393029; sys.cas for mutexes 84ded32817.
  • 2009: G structs recycled 2aea4a063b; list of all Ms kept so the collector does not free them 93689d863c; preemption-by-GC checks in expensive routines 5328df6534 (fixed in 711088106e); runtime.Goroutines 51a2183851; idle goroutine 582fd17e11; contention reduced by waking on scheduler unlock 5e963a826c; scheduling race fixes 774333685e, 88ce9ce40c, 50d6c81d4a; no new threads during malloc 4dfd7fdde5.
  • 2010–2011: GOMAXPROCS vs GC bug 29d78f1243, running-goroutine count bd77619142; faster syscalls 025abd530e, ba134539c5; scheduling race 9bc58accce; unnecessary locking removed 6500065543; bootstrap M counted in mcpu 2572ca2ff2; main goroutine locked to main thread during init 6808da0163 and dc159fabff; runtime.NumCPU 6dfdd4c1e3; deadlock detection restored in the simplest case 84bb2547fb/5eb007dede; goroutine profile and stack dumps e4b02bfdc0. Left open: one global scheduler lock and one run queue; GC stats and goroutine ids still global (the false-sharing fixes 909f31872a, 013ad89c9b are band-aids); no preemption of running goroutines.
• Go 1.1 — Done: user-visible the per-P scheduler 779c45a507 with the preparatory commits above; local queues 353ce60f6e; runtime.Gosched forward progress for locked goroutines f87b7f67b2/a92e11a256; deadlock-detector false negatives 06a488fa97, 6ee739d7e9; clear locked bit on goroutine exit 1a4599b41a; integrated poller wiring c211884731. Left open: no preemption (long-running loops starve others); global queue can be starved (fixed in 1.2).
• Go 1.2 — Done: fairness 4bb491b12e (global runqueue no longer starved when goroutines keep re-readying each other); CPU underutilization fixes 15a1c3d1e4/32fef9908a (newproc/ready were “deliberately sloppy” about waking Ms); starttheworld race dbcfed93e7; sysmon no longer parks while goroutines run 156e8b306d/658d19a53f; singleproc removed d5ab784611; stack size limit 757e0de89f and thread count limit 665feeedcb; LockOSThread fix 4961483e7d; windows priority boost disabled a574822f80. Left open: timers/locks still via notes.
• Go 1.3 — Done: lock-free fixed-size ring for local work queues, overflow to the global queue 4722b1cbd3; per-P defer pool 1ba04c171a; goroutine id batching 98b50b89a8; P retake tuned so the last P in a syscall can be retaken 179d41fecc; fair scheduling during frequent GCs 90eca36a23; goroutines held in an array for parallel root marking cb133c6607; main Goexit semantics 55e0f36fb4, ade6bc68b0. Left open: scheduler still C.
• Go 1.4 — Done: scheduler functions converted to Go 684de04118, 5d40742728, 6ed58c2962, dropg/schedule rework to handle locked goroutines in one place 64c2083ebc; scheduler trace now prints spinning Ms a7186dc303; goroutine ids start at 1 aa76377423; allp static array 858c57f5bd; NumGoroutines O(1) 92c1e72040.
• Go 1.5 — Done: user-visible default GOMAXPROCS = NumCPU 3ffcbb633e; whole scheduler file converted C→Go in dev.cc b2cdf30eb6; runnext semantics “yield time slice to most recently readied G” e870f06c3f; stealing changed so a P that is about to run a just-readied G is not robbed (c4fe503119), runqvictims/copy removed a0fc306023; spinning bug when readying inside mspinning b0b1a66052; panic when idling a P with runnable Gs 2a46f55b35; runqempty factored e5e52f4f2c; procresize simplified ce9a4afa6f; time slice made a const a23a341e10; “do not do futile netpolls” f47e581e02; randomized scheduling in -race 2028077899; sysmon-triggered GC concurrent 4b2774f5ea.
• Go 1.6 — Done: only spinning Ms wake new spinning Ms (fewer thread wakeups) fb6f8a96f2; p.runq as []guintptr 0ca4488cc1; nmspinning comparison fix 3efe36d4c4; handoff P starts an M when only GC work exists (deadlock with locked Gs) f309bf3eef; unnecessary worker wakeups removed; proc1.go merged into proc.go 386fa03609.
• Go 1.7 — Done: improved randomized stealing (the PRNG missed Ps with “frightening probability”: at GOMAXPROCS=3 one P missed 101,620 times per 1e6 tries) ea0386f85f; usleep(100) in runqgrab reduced to 3us, osyield on Windows 3b246fa863, CPU underutilization fcd7c02c70; spinning in scang/casgstatus replaced with yielding 475d113b53; MaxGomaxprocs clamp 31cf1c1779.
• Go 1.8 — Done: first-class Gscan/state docs b1a6e07919^1.7.
• Go 1.9 — Done: MaxGomaxprocs raised to 1024 fcb45b9c61; extra M’s G marked dead when unused 4e7067cde4.
• Go 1.10 — Done: user-visible GOMAXPROCS limit eliminated ee55000f6c; no sleeping before stealing unless the victim P is running 868c8b374d; fractional GC workers on all Ps e09dbaa1de; sched.mnext int64 6c7bea6309; allp race in findrunnable 7c2cf4e779.
• Go 1.12 — Done: manually managed G dequeues replaced by gList 3578918b66 (used in netpoll de990545c3, gfree 8e8cc9db0f, closechan a034f310b0); gchelper eliminated 0906d648aa; scheduling disabled by class 6e9fb11b3a; m.p never stale 8e0ec5ec09.
• Go 1.13 — Done: look for an idle P when switching to a GC worker or trace reader e56c73f17b; initP/destroyP split out of procresize 9d088fbfda; switch to P0 before destroying current P 4598c23c6c; acquirem/releasem used more widely ea9859f858; P statuses documented b0c6165605.
• Go 1.14 — Done: _Grunning Gs always have g.m/g.m.p fc8eb264bb; timers on Ps integrated in the scheduler (see Timers thread) 06ac26279c; preempt stolen/unlocking d80ab3e85a.
• Go 1.15 — Done: goroutines found by the poller go on the local run queue 2fbca94db7; newproc1 no longer starts the goroutine b3863fbbc2; wakep condition moved inside 4e00b4c366; runqputbatch holds sched.lock 6b4dcf19fa; startm/checkdead visibility 85afa2eb19.
• Go 1.16 — Done: idle Ps no longer stolen from, scaling with GOMAXPROCS 4a2cc73f87; timer steal skipped for Ps without timers (timerpMask) fc116b69e2; injectglist critical path reduced 613388315e; worldsema released with direct G handoff 64a9a75ce9; startm without preemption 370682ae98; sched.lock assertions d42b32e321.
• Go 1.17 — Done: findrunnable refactoring in preparation for the 1.18 spinning rework: work stealing in its own function 60ab197bc2, spinning coordination documented 13368ab56a, timer and spinning rechecks reordered f6e7fe2711, dbade774c3, 4d56576ec0, tryWakeP removed 800fb11efb; runqdrain for GC workers fbb600b283/3b304ce7fe; gFree lists moved in batches between P and global 1f7a01459b; pidleget/pidleput under sched.lock 2520e72d3b; exitsyscall0 race e0d029f758; p struct padding 55d7dcc3cd. Left open: the findrunnable commit messages cite #43997 and #44313 (spinning M races) as unresolved.

Preemption: cooperative stack-check preemption (1.2) to signal-based asynchronous preemption (1.14)

Span: Go 1.0 → Go 1.17 · Status at go1.17: both mechanisms active (async preemption on by default, GODEBUG=asyncpreemptoff=1 to disable) → continues as Asynchronous preemption: extended register state (1.18+)

Paths not taken: Dmitry Vyukov’s 2013–14 scheduler experiments (partially landed) · Loop preemption by compiler-inserted checks and page faults (superseded)

Design. Until 1.2 a goroutine only gave up its M voluntarily (channel, lock, syscall, Gosched) or at GC checks in expensive runtime routines (5328df6534). Go 1.2 added cooperative preemption by reusing the stack-split prologue: setting g->stackguard0 = StackPreempt makes the next function call enter morestack, which turns it into a reschedule. The pieces: a preemptone function (354ec51666), sysmon issuing requests after 10ms of running one G (bc31bcccd3), STW issuing requests every 100us (1e112cd59f), and m->locks reset only after the real unlock so a G holding a runtime lock is never preempted (764bb36ea2). Reliability fixes were needed because the flag could be overwritten by stack growth or lost (5887f142a3, f9066fe1c0). The prologue approach cannot interrupt tight loops without calls, and in Go 1.14 Austin Clements added signal-based preemption: suspendG/resumeG generalized stack-scan suspension (3f834114ab), preemptone sends SIGURG (177a36a5dc), and an asyncPreempt assembly stub spills all user-visible registers (a3ffb0d9eb, generated by mkpreempt.go). A PC is async-safe only if it is not in the runtime, not in an assembly function, and has no “unsafe point” marker; unsafe-point marking was switched from stack map index -2 to the register map so calls can be marked (c3f149250e).

Timeline.

• Go 1.1 — Done: nothing yet; Gosched remained the only yield (f87b7f67b2). Left open: issue 543, tight loops starve the scheduler and GC.
• Go 1.2 — Done: user-visible preempt function (354ec51666, not yet used), preemption for GC (1e112cd59f), preempt goroutines running more than 10ms (bc31bcccd3), enabled by default a20784bdaf after temporary disable/re-enable (08e064135d, b913cf84dc, 1da96a3039), debugging knob removed 10ebb84d48, old gcwaiting checks removed f195ae94ca; notetsleepg and notetsleep returning false on timeout for preemptive sysmon e97d677b4e, e932c2035f; sysmon polls network if idle >10ms 6857264457. Left open: loops without function calls cannot be preempted.
• Go 1.5 — Done: preemption used by the concurrent collector’s ragged barrier (see GC interaction thread).
• Go 1.13 — Done: user-visible a goroutine that runs many short syscalls is now preempted (the 10ms rule did not cover them) 4166ff42c0.
• Go 1.14 — Done: user-visible asynchronous preemption: abstract M preemption check d1969015b4, signal-based preemption in preemptone 177a36a5dc (also makes freezetheworld more robust), x86 asyncPreempt a3ffb0d9eb, GODEBUG=asyncpreemptoff=1 40b7455877, NO_LOCAL_POINTERS assembly functions excluded 3873e5497d, Windows support b89b4623eb, register map for unsafe points c3f149250e, innermost-frame function name for the safe-point check dcdee153cd, upper Y-register bits zeroed on darwin/amd64 123f7dd3e1. Left open: stubs only; other architectures not yet filled in as of the first commits (a3ffb0d9eb “stubs out the other arches”).
• Go 1.15 — Done: no preemption signal if one is already pending 0c0e8f224d; preempt-stop cleared in dropm 8de0bb77eb^1.14; cleantimers returns early if the G is being preempted 5d70cb0667; e8be350d78 blocks preemption while a timer is timerModifying.
• Go 1.16 — Done: wait for preemption signals before syscall.Exec 05739d6f17/21cfadf0dc; SSE guard restored in asyncPreempt on 386 67edc0ed81; ignore SIGPROF when profiling disabled 2556eb76c8; riscv64’s g register not saved 76661d12e8.
• Go 1.17 — Done: R15 saved before checking AVX state c49c7a675a; SPWRITE functions in traceback accounted for (not async-preempted since assembly functions never are) 03886707f9; preemption disabled in addtimer aa26687e45.

Network poller: from the netFD select loop to the runtime-integrated epoll/kqueue/IOCP poller

Span: Go 1.1 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Netpoller (1.18+)

Design. Before 1.1 package net ran its own poller goroutine and Windows used a separate IOCP loop. Dmitry Vyukov moved polling into the scheduler: netpoll(block) returns a list of ready Gs; pollDesc objects (one per fd) hold rg/wg pointers that are either nil, pdReady, pdWait or a G pointer, updated by CAS; deadlines use runtime timers. The scheduler polls non-blockingly in findrunnable, sysmon polls if nobody has for 10ms (6857264457), and an idle M can block in epoll_wait/kevent instead of a futex. The invariant “there must be at least one running P or a thread polling the network” (0b5d55984f) prevents netpoll deadlocks. internal/poll (1.9) then factored the fd lifecycle (fdMutex) out of net and the os package began using the poller for pipes/files (c05b06a12d).

Timeline.

• Go 1.1 — Done: user-visible scheduler hooks c211884731, d0c11d20b8 (timer prototypes), add3349867 (xchg64), integrated poller for darwin 0bee99ab3b (TCP4Persistent 67786 → 33175 ns vs old poller), Linux/ARM 1d64d04da5, explicit removal of fds from the epoll set before close (#5061) 44840786ae, deadlock fix 0b5d55984f; Windows band-aid giving a blocking hint to GetQueuedCompletionStatus ea15104110. Left open: Windows and Solaris not integrated.
• Go 1.2 — Done: Windows poller moved into the runtime 6ea7bf253c with GetQueuedCompletionStatusEx batching 65834685d3; Solaris (event ports, level-triggered) 50df136483^1.3; netpoll refactors 8486d96a27, 2ea859a779^1.3 (level-triggered IO); FreeBSD/ARM e82614e5be; deadlines for already-issued IO aaab946943; netFD fdMutex 23e15f7253; fewer allocations per IO 04b1cfa946; netpollready called once per event 82ef961af5.
• Go 1.3 — Done: Solaris use-after-close race 199e703083.
• Go 1.4 — Done: netpoll converted to Go (epoll/kqueue) 91a670d179, Solaris 97fdfdb52c; shrinkstack vs netpoll race 2b62e1eaec^1.5 (Go 1.5 list).
• Go 1.5 — Done: spurious deadlock in netpoll 776aecaf6e; futile netpolls skipped f47e581e02; Windows netpoll converted 0438182c30.
• Go 1.6 — Done: throw on unexpected netpoll error instead of looping b6d115a583.
• Go 1.9 — Done: internal/poll refactor 3792db5183 and os file I/O through the poller c05b06a12d (1.9); netpoll panic messages 91c9b0d568.
• Go 1.10 — Done: only call netpoll if netpollinited 86cd9c1176; errno positive in netpollopen 4a0dcc2de1.
• Go 1.12 — Done: deadline timers merged: modtimer used by netpoll a86f549703, no re-creation of unchanged timers 86d375498f, memory barrier only if a timer fired 31e7842f3d, StorepNoWB 56421f26ef, no runtimeNano on the common path (negative duration = expired) 21f7f01289; nanotime wrappers moved to time/poll ff51353c38.
• Go 1.13 — Done: user-visible event-scanning errors reported on read events a5fdd58c84, narrowed to critical errors only 451a2eb0ab, disabled on Solaris e0181ff52a.
• Go 1.14 — Done: netpoll takes a delay for blocking 831e3cfaa5 (needed by timers on Ps); netpollBreak for AIX 301bc66a47; wake netpoller when dropping a P with timers d80ab3e85a; port_getn ETIME 971ec8728e; init made complete before return 04e80fb2c9, 9e094ea01e.
• Go 1.15 — Done: converge duplicate netpollBreak calls (89% faster) 0cc1290174; Windows always uses GQCSEx e3cf0525b0; AIX break races 78a45d8b45, 45f99d85e0; netpollWakeSig uint32 245409ea86; error code names d98023ebb5; pollDesc/netpollunblock refinements 33357270f1.
• Go 1.16 — Done: netbsd netpoll overrun detection in sysmon cb95819cf6.
• Go 1.17 — Done: netpollopen error handled in poll_runtime_pollOpen ebb92dfed9, netpollclose documented 437d229e2a, pollDesc lock partial edges e8e425cb23, 0e0a872a76^1.16.

Timers: timerproc goroutine → per-bucket heaps → timers on Ps

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (timers in per-P heaps, driven by findrunnable/checkTimers) → continues as Timer rewrite: per-P timer heap, timer.lock, and GC-able synchronous timer channels (1.18+)

Paths not taken: Dmitry Vyukov’s 2013–14 scheduler experiments (partially landed)

Design. Timers began as a single timerproc goroutine with one global heap (3b860269ee “add timer support, use for package time”; bug fixes dc6726b37f, a899a467f2). Go 1.2 made the heap 4-ary (fcf6a7e5ce), used a monotonic clock for timers on linux (86c976ffd0) and added tsleep overflow handling (3548ab5ebb). In Go 1.10 Aliaksandr Valialkin’s 76f4fd8a52 (“improve timers scalability on multi-CPU systems”) sharded the heap into timersBuckets (one per P, timersLen buckets), each with its own goroutine; goroutines still ran timerproc and had to be woken to run timer functions.

Go 1.14 (Ian Lance Taylor, tracked as #6239 and #27707) rewrote this to put timers on Ps. Each P owns a heap of *timer, a status word timerNoStatus/timerWaiting/timerRunning/timerDeleted/timerRemoving/timerRemoved/timerModifying/timerModifiedEarlier/timerModifiedLater/timerMoving is advanced by CAS and timer.pp points to the owning P. The scheduler calls checkTimers from schedule and findrunnable, which runs expired timers inline on the P (no goroutine), adjusttimers applies modified timers, cleantimers/clearDeletedTimers drops deleted ones lazily, and idle Ps wait in netpoll with a deadline set to the next timer so that time.Sleep and netpoll share one wakeup mechanism. The new code was landed as ~15 bisectable commits that were added but unused until the switch (2e0aa581b4 addtimer, 7416315e33 deltimer, 48eb79ec21 modtimer, eff3c1e426 resettimer, 9093b1def0 dodeltimer, 4665470147 cleantimers, 7d891d5e4d movetimers, 220150ff3c adjusttimers, 432ca0ea83 runtimer), then 6becb03334 switched (“AfterFunc 1.57ms → 0.07ms, After 1.63ms → 0.11ms”) and 580337e268 removed the old code. Later stages were stealing timers from running Ps, avoiding scans and cutting latency.

Timeline.

• Go 1.0 — Done: runtime-provided timers for package time 3b860269ee, timer callbacks crash dc6726b37f, timer removal corruption (#2495) a899a467f2. Left open: one global heap and goroutine.
• Go 1.2 — Done: 4-ary heap fcf6a7e5ce, monotonic clock on linux/386, amd64 86c976ffd0^1.3, tsleep overflow 3548ab5ebb, fewer notewakeups in timerproc d1b66439f9, network timers crash 9707f269c1.
• Go 1.3 — Done: timerproc clears f/arg to avoid leaks be1c71ecb5.
• Go 1.4 — Done: timers converted to Go 9601abaf8b; fake time support revived 2fe9482343.
• Go 1.9 — Done: time.Sleep no longer allocates a1261b8b0a; “use next timer to decide whether to relax” 093adeef40.
• Go 1.10 — Done: per-P timer buckets 76f4fd8a52; sift functions optimized 6a7c63a08a; early return from deltimer a72e26f246; monotonic zero on low-resolution clocks 986582126a.
• Go 1.12 — Done: timerproc not woken needlessly a9280fa2de; startNano refactor a1ee0a21cf.
• Go 1.14 — Done: user-visible timers on Ps 06ac26279c (scheduler), 6b3bb4ba3b (procresize/destroy), a7ce2ca52f (resettimer), 0050c079d5, the add/del/mod functions above, 7d84245a9c (Sleep), c824420d47 (timeSleepUntil), ab3f1a23b6 (race), 10e7bc994f (wasm), switch 6becb03334, removal 580337e268; fix-ups e96fd13264, dc39be8b85, c3cef0bfe5 (timerMoving), b50fcc88e9, e762378c42, 2d8c1995b9 (release timersLock while running a timer; deadlock with the scavenger lock), 94f4686a77, 641e61db57 (remove deleted timers when >1/4 of a P’s heap), cfe3cd903f (atomic first-timer field). Left open: timers on running Ps cannot be stolen; GOMAXPROCS shrink hazards.
• Go 1.15 — Done: user-visible time.Ticker.Reset 6e5652bebe/402ea9e4f9; timers on running Ps can be stolen 9b90491e4a; racy timer use no longer panics 98858c4380; resetTimer only from Reset 5f3354d1bf; wakeNetPoller avoided in timerModifying 355f53f0a0; timer/event-handler race 0452f9460f.
• Go 1.16 — Done: timer latency fix: expired timers treated like steal-able work 8fdc79e18a; adjust only when needed b4b0144652; skip timer stealing for timerless Ps fc116b69e2, 89a6540d8a, e02ab89eb8; when and period strictly validated b78b427be5, b635e4b808; timerWaiting P field fix d2d155d1ae.
• Go 1.17 — Done: findrunnable timer computation moved f6e7fe2711; adjustTimers counter removed bfbb288574, 798ec73519; addtimer without preemption aa26687e45; time.now in assembly on linux/amd64 d09947522d with walltime removed 42953bc9f5, b15372f39b, c96fec9036.

Channels and select

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (hchan with a mutex, sudog queues, selectgo with lock-ordered cases) → continues as Scheduler core: spinning Ms, work conservation, stealing, STW interaction (1.18+)

Paths not taken: Lock-free channels and finer-grained select locking (never landed)

Design. Channels date from July 2008 (594175d0b5 “chan”, 526200345c synch chan, 2d9ff40774 asynch), select from e963cba8a1, select default from b69e80d8dd, and close in 4523ee9ac8. In the C runtime the channel had a single lock per chan (global chanlock removed, d1740bb3a6), a circular buffer (replacing a circular linked list, 507df959e4) and SudoG wait queues. 2011 performance work (Vyukov): SudoG always points to the element and sits on the stack, copying and wakeups moved outside critical sections 90f3cb13fb; chan creation without finalizer on futex platforms (66% faster) d770aadee5; specialized type algorithms 54e9406ffb.

Select picks a random permutation (pollorder) to be fair (4f269d3060) and locks channels in address order (lockorder). A blocked select queues a sudog on every channel with a shared selectdone word that waking goroutines CAS (the stack-growth race in this protocol was fixed in 1.8 b902a63ade). Go 1.4 moved chan and select into Go; Go 1.5 made sudog linked lists doubly linked, giving O(1) dequeue instead of O(n^2) on popular channels (BenchmarkChanPopular 2.56ms → 0.63ms, 8eb8b40a49); Go 1.6 simplified buffered operations so that a waker completes the whole protocol with its counterpart and the sleeper never re-locks (single copy sender→receiver, 8e496f1d69, relanded as e410a527b2 after a revert); Go 1.7 made sudog.elem protected by hchan lock and recorded the channel in the sudog to support concurrent stack shrinking (e4a95b6343, db72b41bcd, 276b177771).

Timeline.

• Go 1.0 — Done: chan/select/close (above); select fairness 4f269d3060 and memory leak pass 3 ec53627ed5; channel data cleared on receive for GC dec248c665; make(chan [0]byte) 95323c59ea; sync channels faster 90f3cb13fb; x, ok = <-c zero value d16bc7a9f2; disallow close on receive-only channels f58ed4e641; heap sort replaces bubble sort in select (1.1) cb32ea9c19^1.1; lockorder before pollorder 53e139c7a0.
• Go 1.1 — Done: precise GC of channels a656f82071; heapsort select lock order cb32ea9c19.
• Go 1.2 — Done: close/send race detection in select 62747bde6c; chan blocking profile incl. select and close c92287686d; closechan stack split 10d1e55103.
• Go 1.3 — Done: make(chan) 32-bit overflow d409e44cfb; chan code refactor with selgen local e1ee04828d; reflect passes pointers to the runtime 873aaa59b7; race detection of channel sync 9e1cadad0f, abd588aa83.
• Go 1.4 — Done: chansend/chanrecv/closechan/select converted to Go 9a1e142bbc, 47d6af2f68, 6a76bca33f, 1d8fa7fa5d; non-blocking fast paths (BenchmarkChanNonblocking 27.8 → 7.8ns) f6ceefa2bf; chan type info dfa5a99ebb; sudog stale-pointer leaks 0120f8378d, 13da360845, e330cc16f4, b3932baba4, 2ad99f0960, b3932baba4.
• Go 1.5 — Done: doubly-linked sudog queues 8eb8b40a49; permutation tightened fd5540e7e5, modulus removed 9a0fd97ff3; typed memmove for chan content 54bb4dc390; memory barrier for sync send in select 8c3533c89b; sudog cache bounded 5ef145c809.
• Go 1.6 — Done: simplified buffered channels 8e496f1d69, e410a527b2.
• Go 1.7 — Done: indexes for select lock order 26594c3dfd; g.waiting in lock order 005140a77e; hchan.lock protects sudog.elem db72b41bcd; lock order between G status and channel lock d45bf7228f; sudog documented 857d0b48db.
• Go 1.8 — Done: select/stack growth corruption fix b902a63ade; write barriers on direct receive f2d05389da.
• Go 1.9 — Done: selectgo joined d50f892abc.
• Go 1.10 — Done: selectdone moved into g 32b94f13cf; makechan memory checks 455775dae6.
• Go 1.11 — Done: debugSelect toggle dbd66fd3d0.
• Go 1.12 — Done: races between close and len/cap ignored 83dfc3b001; makechan overflow multiplication check 286c7ae10c.
• Go 1.14 — Done: receive on empty closed channel fast path e1446d9cee (backported as 02e492f1a3^1.15 in 1.15); scasetype 400d021233.
• Go 1.15 — Done: pack sudog struct 94e61ab94d.
• Go 1.16 — Done: sudog success field 30a68bfb80, releasetime removed 78a1064d5d; nil-channel cases omitted from selectgo’s orders 8a984e8e3f; stack shrinking disabled in the activeStackChans race window eb3c6a93c3; race acquire/release order 35455fff0e; df68e01b68.

Semaphores, runtime locks and Notes

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (futex/sema lock, note, semaRoot treap) → continues as Runtime mutex: tri-state futex/sema locks → “spinbit” mutex (1.23+)

Design. The runtime had two primitives: Lock (spin then sleep) and Note (one-shot event with notesleep/notewakeup), plus sync semaphores (semacquire/semrelease, published for sync in 5bbd4c2f1b). Early on each OS had its own mutex; ee24bfc058 (Go 1.0) unified them into two generic implementations, futex-based and semaphore-based, each OS supplying a few callbacks (futexsleep/futexwakeup or semasleep). That removed chan finalizers. Notes gained timed sleep (notetsleep, user-goroutine notetsleepg) for preemptive scheduling (Go 1.2: e97d677b4e, e932c2035f) and extra debugging for double wakeups (4380fa6d99, 98d94b589c).

User-level semaphores live in sema.go as a hash table of semaRoots indexed by address. 1.9 fixed scaling for pathological hashing: Russ Cox first made each root a two-level list (O(n²) → O(n) when n goroutines wait on two locks hashing to one root, 45c6f59e1f), then a balanced treap keyed by address (O(n log n) with n/2 colliding locks, 990124da2a), recorded the mutex-event timestamp before readying (b067ad939d) and moved mutexevent profiling out from under the root lock (58d762176a). cansemacquire/semrelease handoff for starving mutexes (1.14) lets Unlock yield to the waiter (a8f57f4ada, relanded 7148478f1b; trace event a4c579e8f7). Go 1.9 also added a runtime rwmutex type (09ebbf4085) working on Ms instead of Gs (80832974ac), used for execLock (9745e88b22) and requiring that Ms holding a read lock not be descheduled (f3b5a2bc19).

Timeline.

• Go 1.0 — Done: mach semaphores only under contention 5ff12f871f; sync semaphores 5bbd4c2f1b; runtime mutex across OSes ee24bfc058; lock-free stack a5dc7793c0^1.1 (Go 1.1 list). Left open: none noted.
• Go 1.1 — Done: lock-free stack a5dc7793c0, semaphore table hidden from GC 35724c1aa5, double-wakeup diagnosis 4380fa6d99; unions → structs for Lock and Note d617454379, 58030c541b.
• Go 1.2 — Done: notetsleepg e97d677b4e; notetsleep timeout result e932c2035f.
• Go 1.4 — Done: sync and async semaphores converted to Go a6950fe0f9, ebac0e6f30; notes afb2260491, futex notes fix d2165672ca; block-profile fix 7b3677bf3c; semacquire always on the G stack da8cf5438a and 71de7e7539.
• Go 1.5 — Done: write-barrier throw in lock_sema fixed (uintptr) 87ec06f961.
• Go 1.9 — Done: semaRoot two-level list 45c6f59e1f, balanced tree 990124da2a; runtime rwmutex 09ebbf4085, 80832974ac, 9745e88b22, f3b5a2bc19.
• Go 1.11 — Done: semaphore waits distinguished from sync.Cond.Wait in goroutine status e13a213c7f.
• Go 1.13 — Done: AIX noteclear 1956b28ae3; notetsleep sanity check ac40a7fb9e; js/wasm lock implementation cd03664f82.
• Go 1.14 — Done: direct handoff yields to starving waiter a8f57f4ada/7148478f1b; trace event in semaphore handoff a4c579e8f7.

sync.Mutex, RWMutex, WaitGroup, Cond, Once

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as sync package API evolution (1.18+)

Paths not taken: Per-CPU sharded values: sync.Cache, distributed RWMutex, percpu.Sharded (still open)

Design. bf3dd3f0ef added mutex.Mutex in 2008 (first on sys.cas, 84ded32817), 91ceda5c18 a stub RWLock, and 12206f64dd the first real sync.RWMutex (“support for concurrent readers”). sync.Once was added in 895c5db6df, sync.Cond in 05b1dbd0a6, sync/atomic in 22eab1f5c7 with sync rebased on it 12b7875bf2.

sync.Mutex evolution: 2011 RWMutex with a wait-free fast path for readers (BenchmarkRWMutexUncontended 179 → 96.6ns, daaf29cf93); WaitGroup fast paths (−64% uncontended, ee6e1a3ff7). Go 1.5 added active spinning: “limited active spinning” if running multi-core, GOMAXPROCS>1 and at least one other running P with an empty local run queue (edcad8639a). Go 1.9’s 0556e26273 made the mutex fairer: normal mode (new arrivals can steal the lock) vs starvation mode (ownership handed off directly after a waiter waits more than 1ms, new arrivals queue at the tail), and fixed requeueing at the tail of the wait list (waiters that were woken but lost go to the head). Go 1.13 made the Lock/Unlock/Once.Do/RUnlock fast paths inlinable (41cb0aedff, 4c3f26076b, ca8354843e, 05051b56a0), and Go 1.14 added the runtime-level yield to the waiter.

WaitGroup was simplified to a single 64-bit state word (03a48ebe1c removed dynamic semaphores; 9ff7df003d made it more space-efficient in 1.11). Cond went from semaphores-per-waiter (5a20b4a6a9: no allocation, 30–50% faster, copy check) to a notification list (8e7072ca83, 1.7) fixing Broadcast missing waiters.

Timeline.

• Go 1.0 — Done: Mutex, RWMutex rewrite 12206f64dd, Once 895c5db6df, Cond 05b1dbd0a6, Unlock-of-unlocked check 26880d7e03, atomics-based Once and RWMutex fast paths d2ab2b9f42, daaf29cf93, ee6e1a3ff7, Cond.Signal wakes previously sleeping goroutines 17bfa32fde, barrier proposal 63457d089e, armv5 support 86b0ea6447; Once.Do uses atomic Store (1.1) 8c4c6c413f^1.1.
• Go 1.1 — Done: race instrumentation for sync 53390c8fc7, 5bb3a66a97.
• Go 1.2 — Done: Cond without allocation 5a20b4a6a9; WaitGroup race fixes 0f4897ae1a, 07cb48c31f.
• Go 1.3 — Done: WaitGroup spurious wakeup e9347c781b.
• Go 1.4 — Done: RWMutex misuse detection 22d46d53ea; panicking functions still mark Once complete b22dc6385d.
• Go 1.5 — Done: active spinning edcad8639a; WaitGroup simplified 03a48ebe1c.
• Go 1.7 — Done: new Cond 8e7072ca83; RWMutex recursive RLock documented b3f98d7a42.
• Go 1.8 — Done: unlock of unlocked mutex throws, not panics 40d81cf061 (also f72196647e^1.10 for 1.10).
• Go 1.9 — Done: user-visible fairer Mutex with starvation mode 0556e26273.
• Go 1.11 — Done: RWMutex in the mutex profile 88ba645827; WaitGroup space 9ff7df003d; RWMutex race annotation f451a318bb.
• Go 1.13 — Done: inlinable fast paths (above).
• Go 1.14 — Done: yield to waiter on starving unlock a8f57f4ada.
• Go 1.16 — Done: RWMutex race semantics and Once copy documented b4f3d52f6a, f8376a55b0; sync special import case removed c305e49e96.

sync.Pool

Span: Go 1.3 → Go 1.17 · Status at go1.17: still the design at go1.17 (per-P poolLocal with private slot plus poolChain of lock-free dequeues, victim cache) → continues as sync package API evolution (1.18+)

Paths not taken: Per-CPU sharded values: sync.Cache, distributed RWMutex, percpu.Sharded (still open)

Design. Added as a stop-gap by Brad Fitzpatrick (8c6ef061e3, “temporary implementation, until Dmitry makes it fast”) with the registered cleanup in the GC (mgc0.c). A week later f8e0057bb7 made it scalable: P-local caches via procPin (introduced in 90653d7864), batches moved to/from a mutex-protected shared list (BenchmarkPool-16 403311 → 1598 ns); 8fc6ed4c89 limited the local cache to 1 element per P because with GOMAXPROCS=32 and 2MB objects Pool could waste 960MB. Go 1.13 rewrote the shared part: poolDequeue (fixed-size lock-free ring, 2b60567002) chained into poolChain (57bb7be4b1) so that Put pushes at the head and other Ps steal from the tail without locks (d5fd2dd6a1), and poolCleanup was changed from “clear everything on each GC” to a victim cache (2dcbf8b369), so objects survive one more GC. Go 1.17 randomizes the stealing start via randomOrder copied from the scheduler (d5d24dbe41).

Timeline.

• Go 1.3 — Done: user-visible sync.Pool 8c6ef061e3; scalable P-local f8e0057bb7; race detector ce8045f393; 1-element local policy 8fc6ed4c89.
• Go 1.4 — Done: Pool memory released across second and later GCs af3868f187.
• Go 1.8 — Done: Pool enabled under race ba048f7ce4.
• Go 1.9 — Done: faster indexLocal (−47%) af5c95117b, poolLocal aligned to cache line 8aa31d5dae.
• Go 1.13 — Done: lock-free queues 2b60567002, 57bb7be4b1, d5fd2dd6a1, victim cache 2dcbf8b369.
• Go 1.17 — Done: random victim selection when stealing d5d24dbe41.

sync.Map, atomic.Value and sync/atomic

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as sync/atomic and runtime atomics (1.18+)

Design. sync/atomic was added in 2011 (22eab1f5c7) with LoadUintptr, Store and 64-bit Load/Store added later (ea23ba3e2d, 1fc676332f, cf0d8c0941), MFENCE replaced by LOCK XADD for 386 compatibility (ab596cae9e), a Linux/ARM kernel cas64 helper for 64-bit ops (090f9fc3ef, 4d724f4c5b), and run-time checks for 64-bit alignment on 386 and ARM (14a295c817, 94b7853924, and docs 64a0017d6e). Swap functions came in 1.2 (0e15b03f93).

atomic.Value (1.4, 98a1e207e2) stores an interface with the type word written first as a sentinel ^uintptr(0) and published atomically (copy-on-write idiom). sync.Map (1.9) was imported from x/sync/syncmap (959025c0ac): a read-only atomic map with a dirty map and miss counter, expunged entries; 1.15 added LoadAndDelete (2e8dbae85c, fix 94953d3e59) and 1.17 added Value.Swap/CompareAndSwap (2422c5eae5). In the compiler, atomics were intrinsified on amd64 (1.8, d6098e4277) and many sync/atomic functions redirected to runtime/internal/atomic (1.11, 1b6fec862c).

Timeline.

• Go 1.0 — Done: package introduced (above) 22eab1f5c7, ARM cas fixes 540feaae47, 0b200b4d09; atomics on 386 with XADD ab596cae9e.
• Go 1.1 — Done: Linux/ARM 64-bit atomics 090f9fc3ef, unaligned detection 14a295c817, 94b7853924; FreeBSD/NetBSD ARM ace9ff4578, 4d8b1feb79; race instrumentation 59b8745328, 27087022ce, f3c1070fa4^1.2.
• Go 1.2 — Done: user-visible Swap functions 0e15b03f93, 883530c019, 66c58cea67.
• Go 1.4 — Done: user-visible atomic.Value 98a1e207e2; nacl/arm 62af04c0bc; missing prototypes 735289ff76.
• Go 1.5 — Done: pointer write barrier on atomic stores 7b4df8f018; arm64 2a0833da50, darwin/arm a1457cac9c, openbsd/arm 6900a421a4, power64 56949d33d8, memory barriers in arm cas 4bd8040d47.
• Go 1.6 — Done: mips64 d58081cf0a.
• Go 1.7 — Done: plan9 arm 5c613e9162; s390x 7da42d7597.
• Go 1.8 — Done: mips 9788e3d4d7; amd64 intrinsics d6098e4277.
• Go 1.9 — Done: user-visible sync.Map 959025c0ac; alignment doc fea7c43ea2.
• Go 1.10 — Done: Value without noCopy 0c0c3c186b.
• Go 1.11 — Done: atomics redirected to runtime/internal 1b6fec862c, 0a4b962c17.
• Go 1.12 — Done: 32-bit MIPS alignment ca7c12d4c9; Map simplification 756c352963.
• Go 1.15 — Done: user-visible Map.LoadAndDelete 2e8dbae85c; panic64 removed a4c48d61f5.
• Go 1.17 — Done: user-visible Value.Swap and CompareAndSwap 2422c5eae5.

Runtime lock ranking (static lock order checking)

Span: Go 1.15 → Go 1.17 · Status at go1.17: opt-in via GOEXPERIMENT=staticlockranking; hand-maintained partial order → continues as Runtime lock ranking (1.18+)

Design. Dan Scales added lockrank.go (0a820007e7), a documented total order of runtime lock classes. With the experiment enabled, lock2 calls lockWithRank/acquireLockRank which keeps the M’s held-lock list and throws when a lower-rank lock is acquired while holding a higher one; a partial-order table (lockPartialOrder) allows incomparable pairs. It was made explicit that “ordering direction” runs from lower to higher rank (646b4ac065) and that Gscan acquire/release participates (f9640b88c7). Michael Pratt (Go 1.16) added assertions that required locks are held (assertLockHeld, assertRankHeld, 53c9b9588a), sched.lock assertions (d42b32e321) and in 1.17 a check that the partial order itself is consistent (c41bf9ee81). Edges were added incrementally whenever a real nesting was found (1067ab0808, 300ff5d8ac, a81bc8e825, 0f8fecaba7, f474e9e549, e05695ed4d; 1.16 2013f70256, 2754d91164, c68745b130, 0e0a872a76, a69cda9a65; 1.17 e8e425cb23, d75fbac54d).

Timeline.

• Go 1.15 — Done: static lock ranking 0a820007e7, partial-order edges 1067ab0808, 300ff5d8ac, a81bc8e825, 0f8fecaba7, f474e9e549, e05695ed4d, Gscan ordering f9640b88c7, explicit direction 646b4ac065; the experiment itself arrives via 095ba22597^1.17 (GOEXPERIMENT parsing simplification, 1.17). Left open: not enabled by default and requires the experiment build.
• Go 1.16 — Done: held-lock assertions 53c9b9588a, sched.lock assertions d42b32e321, nosplit dropped from primary functions 989ab8a7d6, newstack panic at acquireLockRank fix b246c0e12f, tighter systemstack assertions 420c68dd68, new partial edges (above).
• Go 1.17 — Done: partial order self-check c41bf9ee81; pollDesc edges e8e425cb23; forceGC to traceStackTab edge d75fbac54d.

Runtime from C to Go: scheduler, channels, semaphores, timers, netpoll

Span: Go 1.3 → Go 1.5 · Status at go1.17: completed in Go 1.5; no C in the runtime scheduler

Design. Before 1.4 proc.c, chan.goc, sema.goc, time.goc, netpoll.goc, select.goc were Plan 9 C. Starting with 1.3’s precise stacks the runtime prepared to drop C stacks, in 1.4 Dmitry Vyukov and Russ Cox converted pieces by hand into Go (.go with //go:nosplit, systemstack/onM calls to jump to g0), and in dev.cc Russ ran an automated C-to-Go translator. Consequences visible in the history: functions that park must run on the G stack (da8cf5438a, 71de7e7539); sudogs stopped living on the stack, so a sudog cache was introduced and leaked stale pointers until b3932baba4; write barriers appear in scheduler code so guintptr/muintptr pointer-free types were introduced (eafc482d4f, 239273d963, 0ca4488cc1, 165c15afa3, 12ec54724b) plus //go:nowritebarrier(rec) checks (402e37d4a9, d2c81ad847, 9897e40811).

Timeline.

• Go 1.3 — Done: stack management refactor 1a89e6388c, stack copying toggled f50a87058b, precise stacks ecf700b5ee/53061193f1.
• Go 1.4 — Done: common scheduler 684de04118, notes afb2260491, chan 9a1e142bbc, 47d6af2f68, 6a76bca33f, semaphores a6950fe0f9, ebac0e6f30, timers 9601abaf8b, select 1d8fa7fa5d, netpoll 91a670d179, cgocall c08d8834dd, forcegc helper 42486ffc5d, cpuprof ac49e6735b, mprof 548d0805f7, mallocgc bffb0590c1, funcPC 0e07f1c99f, getgcmask, Gosched 5d40742728, newproc1 on M stack 6ed58c2962, runtime.init run 1d0c89a454, goroutine start/stop at safepoints 56bd176e1d, entersyscall safe for stack splits 5ea69978fd.
• Go 1.5 — Done: dev.cc: headers fee9e47559, scheduler b2cdf30eb6, Windows netpoll 0438182c30, fc288681cf, missing consts e5d01a5ffc; write barriers on typed chan/map/interface copies 54bb4dc390, 9b638bf1bf.

Goroutine and M lifecycle: caches, ids, LockOSThread, Goexit, cgo extra Ms, deadlock detection

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Goroutine lifecycle and identity (1.20+)

Paths not taken: Main-thread APIs: RunWithMainThreadOnly to mainthread.Do (still open)

Design. G structs are recycled via per-P gfree lists (6cdfb00f4e, 779c45a507); 1.7 split the free list into with-stacks and without-stacks (1a2cf91f5e); 1.17 moves them in batches between P and global (1f7a01459b). Goroutine ids were widened to 64 bits (320df44f04), allocated in batches per P (98b50b89a8), start at 1 (aa76377423). Bounded caches: defer pool and sudog cache have fixed-size local pools plus a central pool because “unbounded local caches are bad and grow to infinite size” (8059071153, 5ef145c809, b759e225f5).

LockOSThread was a single bit; in 1.10 it nests (c85b12b579, #20458), with locked threads terminated if the goroutine exits locked (4f34a52913), no thread creation from locked threads (2595fe7fb6, delegated to a template thread) and lockedExt preserved (d0f8a7517a). runtime.Goexit from main: program continues until other goroutines finish, then crashes (55e0f36fb4, ade6bc68b0). Cgo callbacks from non-Go threads use “extra Ms” (needm/dropm); 1.7 allocates as many extra Ms as needed (50048a4e8e), and 1.10 marks the extra G dead when unused (4e7067cde4). Deadlock detection (checkdead) works from mcount - nmidle - nmidlelocked, handles cgo (34c67eb24e) and Goexit-in-main.

Timeline.

• Go 1.0 — Done: G recycling 2aea4a063b, deadlock detection 84bb2547fb, 5eb007dede, LockOSThread 218c393029, main thread locking 6808da0163, GC stats per-M d839a809b2^1.1 (1.1 list).
• Go 1.1 — Done: 64-bit goids 320df44f04; Gosched for locked Gs f87b7f67b2; checkdead false negative 06a488fa97; locked bit cleared 1a4599b41a; park refactor f20fd87384.
• Go 1.2 — Done: deadlock in cgo programs 34c67eb24e; Goexit during init 14062efb16; false deadlock 1da1030b5d; traceback of running goroutines via freezetheworld 01f1e3da48.
• Go 1.3 — Done: Goexit-from-main semantics 55e0f36fb4, ade6bc68b0; id batching 98b50b89a8.
• Go 1.4 — Done: ids start at 1 aa76377423; goroutine locked marker in crash report 07f6f313a9; per-P defer pool 1ba04c171a^1.3 (1.3).
• Go 1.5 — Done: bounded defer pool, sudog cache 8059071153, 5ef145c809, b759e225f5; system goroutines hidden from traces 59495e8dfd; freezetheworld with gomaxprocs=1 5a828cfcde.
• Go 1.6 — Done: NumGoroutine/Stack agree on excluding system goroutines c5bafc8281, fac8202c3f.
• Go 1.7 — Done: gfree split 1a2cf91f5e; extra Ms 50048a4e8e; cgo SIGPROF traceback 84e808043f.
• Go 1.9 — Done: extra M’s G dead when unused 4e7067cde4; stackAlloc removed 0993b2fd06.
• Go 1.10 — Done: nested LockOSThread c85b12b579, no thread creation from locked threads 2595fe7fb6, terminating locked threads 4f34a52913, lockedg/lockedm as guintptr 165c15afa3, lockOSThread not for every cgo call 332719f7ce.
• Go 1.11 — Done: tracebackancestors d9b006a705; waitreason as uint8 4eea887fd4, 4d7cf3fedb.
• Go 1.12 — Done: lockedExt stays when G exits d0f8a7517a.
• Go 1.14 — Done: _Grunning invariants fc8eb264bb.
• Go 1.15 — Done: debug call injection on a new goroutine 3633d2c545.
• Go 1.16 — Done: lockedg status dump bc0b198bd7.
• Go 1.17 — Done: zero-sized frame check moved to newproc1 352d329c44; ncgocall made a global counter d01bc571f7.

Stop-the-world, safe points and GC–scheduler interaction

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Scheduler core: spinning Ms, work conservation, stealing, STW interaction (1.18+)

Design. In the early runtime STW was stoptheworld via sched.gcwaiting and ms parking at the next scheduler entry; 1e112cd59f made STW issue preemption requests every 100us. The concurrent collector of 1.5 brought several scheduler primitives: forEachP runs a safe-point function on each P (f0dd002895 separated its counters from STW’s stopwait/stopnote; edfc979725 runs the function before entering _Psyscall), ragged barriers replace a STW for enabling write barriers (57afa76471, bb6320535d), utilization-scheduled background mark workers with gcBgMarkWorker Gs (8d03acce54, e0c3d85f08, e09dbaa1de), and 4b956ae317 starts GC promptly when the trigger is hit. The G status gained _Gscan bits and states are documented (b1a6e07919). Stack shrinking was made safe concurrently with channel operations by locking channels and tagging sudogs (276b177771, 8fb182d020, 36a432f27b), stack rescan was O(# dirty stacks) (2a889b9d93) until the hybrid barrier made rescan unnecessary (c5ebcd2c8a). Go 1.14 introduced suspendG/resumeG (3f834114ab) replacing scang/restartg, adding _Gpreempted, and removed old stack scanning and gcscanvalid (1b79afe460, 8c5861576a). 1.14/1.15 worldsema is no longer held across mark phase (7b294cdd8d; relanded f1f947af28), with GOMAXPROCS, StartTrace and StopTrace still excluded. In 1.16 the GC workers are managed by a global pool rather than attached to each P (e1faebe7b4).

Timeline.

• Go 1.0 — Done: GOMAXPROCS vs GC fix 29d78f1243; GC in stoptheworld with parallel GC f18e4e44a3, 85916146ea.
• Go 1.1 — Done: helpgc refactor 01826280eb, parallel for 95643647ae, per-thread work buffers d4c80d19a8.
• Go 1.2 — Done: preempt goroutines for GC 1e112cd59f; stoptheworld race dbcfed93e7.
• Go 1.3 — Done: lock-free work queue ring 4722b1cbd3; GC roots parallel cb133c6607; fair scheduling during frequent GCs 90eca36a23.
• Go 1.4 — Done: start/stop individual goroutines at GC safepoints 56bd176e1d.
• Go 1.5 — Done: dev.garbage concurrent scan 6184f46ea3, d42328c9f7, write barriers d3d60975b4; background mark 8d03acce54, ragged barrier 57afa76471, bb6320535d, prompt start 4b956ae317, no write barriers during STW 41dbcc19ef; forEachP f0dd002895, a1da255aa0 (STW/start factoring), 277acca286 (worldsema while starting), edfc979725, e42413cecc, ; controller diagnostics dc870d5f4b, f244a1471d; sysmon-triggered GC concurrent 4b2774f5ea; stack barriers 3f6e69aca5, 840965f8d7, ad731887a7, 87f97c73d3.
• Go 1.6 — Done: stack barrier lock for traceback f90b48e0d3, 3f22adecc7, d446ba99a4; mark workers attach to P after parking eb3b1830b0, 09940b92a0; P handoff starts M for GC work f309bf3eef; forced GC only when not running 01baf13ba5; idle mark workers not started when barriers cleared 20f276e237.
• Go 1.7 — Done: concurrent-safe shrinkstack 276b177771, gopark never passed stack pointers 8fb182d020, O(# dirty) rescan 2a889b9d93, 5b765ce310.
• Go 1.8 — Done: write barrier before pointer write 8f81dfe8b4, no barriers in dropg/save 8044b77a57, c3163d23f0, stack-to-stack copy shading ee785f03a2; spare Gs 2be3ab4415.
• Go 1.9 — Done: rescan list removed c5ebcd2c8a.
• Go 1.10 — Done: assist credit flush on goroutine exit f10d99f51d; accurate STW duration 250a9610a4.
• Go 1.12 — Done: gchelper eliminated 0906d648aa; scheduling disabled by class 6e9fb11b3a.
• Go 1.14 — Done: suspendG/resumeG and _Gpreempted 3f834114ab; old stack scan code removed 1b79afe460; gcscanvalid removed 8c5861576a; worldsema not held across mark 7b294cdd8d; per-P page and span caches a2cd2bd55d, 4517c02f28.
• Go 1.16 — Done: gcBgMarkWorker global pool e1faebe7b4; stack shrinking race window eb3c6a93c3.
• Go 1.17 — Done: runqdrain for GC mark workers fbb600b283/3b304ce7fe.

Scheduler diagnostics: schedtrace, GODEBUG, execution tracer, block/mutex profiles

Span: Go 1.1 → Go 1.17 · Status at go1.17: still the design at go1.17

Design. GODEBUG was introduced in Go 1.2 (4b536a550f: replaces GOGCTRACE; “the plan is to extend”) and gained schedtrace=N (ms period) with scheddetail=1 for per-P/M/G dump (4f2e382c9f; e.g. SCHED 1001ms: gomaxprocs=4 idleprocs=3 threads=6 ...), later with spinning-thread counts (a7186dc303) and delay-overflow fix (f939ee13ae). Block profiling (pprof “goroutine blocking profile”) came in 1.1 (4cc7bf326a) and was extended to channel/select/close (c92287686d). Go 1.5 added the execution tracer (f30a2b9ca7, 5288fadbdc) with explicit per-event sequence numbers (80c98fa901), switched in 1.7 to per-goroutine sequence numbers because a global sequencer slowed parallel machines “up to 10x” (a3703618ea). 1.8 added the mutex profile (ca922b6d36) and 1.11 included RWMutex (88ba645827). Tracebacks show creator PCs (324cc3d040, c33d490020), blocked durations (c0b9e6218c), and ancestors (d9b006a705).

Timeline.

• Go 1.0 — Done: creation pc in traceback 324cc3d040; simplified stack traces 03e9ea5b74; per-pause GC stats bcd910cfe2; profiling thread creation 5b93fc9da6.
• Go 1.1 — Done: user-visible blocking profile 4cc7bf326a; fatal-throw full stack dump 81221f512d.
• Go 1.2 — Done: user-visible GODEBUG 4b536a550f, schedtrace/scheddetail 4f2e382c9f, creator in running traceback c33d490020, block profile for channels c92287686d.
• Go 1.3 — Done: blocked duration in dump c0b9e6218c; invalidptr setting.
• Go 1.5 — Done: execution tracer f30a2b9ca7, 5288fadbdc, sequence numbers 80c98fa901, e72f5f67a1, 26baed6af7; GODEBUG memprofilerate 3c4be235be, 27aed3ce68.
• Go 1.7 — Done: per-goroutine trace sequence a3703618ea; schedtrace blocked field 25a609556a.
• Go 1.8 — Done: user-visible mutex profile ca922b6d36.
• Go 1.9 — Done: runtime/pprof labels definitions 62956897c1.
• Go 1.11 — Done: tracebackancestors d9b006a705; RWMutex profiling 88ba645827.
• Go 1.16 — Done: user-visible GODEBUG=inittrace=1 7c58ef732e.

Fast random numbers for the scheduler

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (xorshift64+ fastrand) → continues as Fast random numbers for the scheduler (1.18+)

Design. fastrand (per-M) first used a trivial LFSR; it was seeded from cputicks to blunt hash-table complexity attacks (8e765da941, RDTSC on x86). Per-M generators avoided false sharing (909f31872a). The LSFR had strong correlation in higher bits (visible in fastrandn, and in stealing, which missed Ps with high probability), so Go 1.9 widened to 32 bits (663226d8e1), added a multiply-shift fastrandn (46a75870ad, ~2x faster), and Go 1.10 moved to xorshift64+ (e7e4a4ffa3, passes Diehard/SmallCrush); 9d9722a2e1 fixed use inside sema.go. Race mode additionally randomizes scheduling (2028077899) and package init order (8515d9cf65).

Timeline.

• Go 1.0 — Done: cputicks seeding 8e765da941, no false sharing 909f31872a.
• Go 1.5 — Done: randomized scheduling under -race 2028077899.
• Go 1.9 — Done: 32-bit values 663226d8e1; faster fastrand()%n 46a75870ad.
• Go 1.10 — Done: xorshift64+ e7e4a4ffa3; sema fix 9d9722a2e1.
• Go 1.13 — Done: randomized package initialization order in race mode 8515d9cf65.

GOMAXPROCS, thread and resource limits

Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as GOMAXPROCS container awareness and dynamic updates (1.25+)

Design. GOMAXPROCS counted CPUs rather than threads from 2008 (efc86a74e4), made callable at runtime (7955490de2, with a hack to return the old value eb48bfbbda) and needed to wait for the main goroutine in init (aa1aaee7fd). Go 1.1 made it the P count and procresize resizes Ps; Go 1.2 added limits: stack size 757e0de89f (default 1GB/250MB) and thread count (665feeedcb, default 10000), later SetMaxThreads overflow fix (10560afb54). Go 1.5 defaulted to NumCPU (3ffcbb633e) and 1.6 documented that NumCPU doesn’t change (c5a94ba24f), 1.7 clamps the OS value to _MaxGomaxprocs (31cf1c1779), 1.9 raised the cap to 1024 (fcb45b9c61) and 1.10 removed it (ee55000f6c).

Timeline.

• Go 1.0 — Done: GOMAXPROCS semantics efc86a74e4, 7955490de2, eb48bfbbda; NumCPU 6dfdd4c1e3.
• Go 1.2 — Done: user-visible stack and thread limits 757e0de89f, 665feeedcb.
• Go 1.5 — Done: user-visible default GOMAXPROCS = NumCPU 3ffcbb633e; NumCPU doc 2de67e9974.
• Go 1.7 — Done: clamp 31cf1c1779, 1024 fcb45b9c61^1.9, unlimited ee55000f6c^1.10.
• Go 1.14 — Done: GOMAXPROCS may not run during mark (worldsema) 7b294cdd8d.

Changes in this slice that are not part of a thread are listed on the release pages: 1.0 · 1.1 · 1.2 · 1.3 · 1.4 · 1.5 · 1.6 · 1.7 · 1.8 · 1.9 · 1.10 · 1.11 · 1.12 · 1.13 · 1.14 · 1.15 · 1.16 · 1.17

Runtime: scheduler, timers, channels, locks and synchronization (Go 1.18–1.28)

Reader: Claude Sonnet. Cites 530 of 688 commits; nothing non-noise is left uncited. Read the design-heavy messages plus about 70 others; the remaining commits were placed from their subject lines.

Timer rewrite: per-P timer heap, timer.lock, and GC-able synchronous timer channels

Span: Go 1.18 → Go 1.28 · Status at HEAD: synchronous (unbuffered-semantics) timer channels are the only mode since asynctimerchan GODEBUG was removed in 1.27; micro-optimisation of channel-timer polling continues in 1.28

Paths not taken: Concurrent and global timer heaps (2024) (never landed)

Earlier history: Timers: timerproc goroutine → per-bucket heaps → timers on Ps (Go 1.0 → Go 1.17)

Design. Each P owns a timers struct (heap of {timer, when} pairs plus a minWhenHeap/minWhenModified hint used by fast paths) and the scheduler checks it from findRunnable/schedule (runtimer, adjusttimers, cleanHead). Until 1.22 every timer carried a CAS-driven status word with many states (timerNoStatus, timerWaiting, timerRunning, timerDeleted, timerRemoving, timerRemoved, timerModifying, timerModifiedEarlier/Later, timerMoving), so deletion was lazy (“zombies” left in the heap and a deletedTimers count). In Go 1.23 Russ Cox replaced this with a real per-timer mutex plus a few state bits: real deadline always in t.when; timerHeaped if in a heap; timerModified if the heap copy is stale; heap deadline lives in the heap slice itself (bb7a29991c). timers.adjust was rewritten from an O(n) loop plus n O(log n) re-insertions into “mark dirty then re-heapify” in O(n) with no allocation and no multi-timer locking (0728e2b139).

On top of that, the user-visible semantics changed: timers and tickers whose channel is unreferenced become collectable even if still running, because a channel-timer is only in the heap while some goroutine is blocked on its channel (508bb17edd); and Stop/Reset guarantee that no stale value is received afterwards. The implementation models “unbuffered channel fed by a coordinating goroutine”: the real 1-element buffer is hidden (len/cap(t.C) report 0), and Stop/Reset take timer.sendLock, discard a pending buffered send, and report “was active” correctly via an isSending marker (966609ad9e). GODEBUG=asynctimerchan=1 restored the old behaviour until its removal (a5bc606a68). Receives on an idle channel-timer run the timer inline (maybeRunChan) instead of needing a heap entry.

Timeline.

• Go 1.18 — Done: fixes in the old CAS-state machine: adjustTimers backs up as far as needed (2da3375e9b). Left open: nothing noted.
• Go 1.19 — Done: timerModifiedEarliest cleared when the last timer is deleted (c006b7ac27). Left open: nothing noted.
• Go 1.20 — Done: timer fields converted to atomic types: p.timer0When, p.timerModifiedEarliest, p.numTimers, p.deletedTimers, timer.status (e1fd51e076, bb5d656a3a, c82bbc0e8e, 6e74c4116a, f78efc0178); schedt.pollUntil likewise (0fc774a68f). Left open: the timer state machine itself is still CAS-based.
• Go 1.23 — Done: Cox’s stacked refactor, deliberately in tiny steps: delete addtimer/clearDeletedTimers (4a3c3ec996, 77f0bd01fb), merge statuses (b2bd7ebafc, 09bfea95cf, 3be36e9b20, ed0ad591d3, db8c6c8c7a), introduce timer.lock and use it in deltimer/modtimer/cleantimers/runtimer/adjusttimers/moveTimers (51df232b12, 2fb5ef889b, 58911599e8, d1e8dc25ff, 1df6db8e4f, c6888d9264) with a new lock rank (b5a99eaa94), status→state and method-ification (a155a2f8a1, 370c8e983b), per-P timers struct (adc575e64c; the message doubts the pair of atomic int64 deadlines hide missed wakeups), then the state cleanup above (6135651867). user-visible GC of unstopped timers (508bb17edd) and synchronous Stop/Reset (966609ad9e). Follow-ups: use of per-M/P timers needs acquirem before choosing ts (lost-sleep flake 2965dc9895), zombie count fixes (40cc3eb278, e38be310a4), timers.wakeTime race (85bbb12141), Ticker.Reset spurious race (24fa7544b0), adjust fast path (1c864333ca), avoiding netpoll wakeups at timer creation (84b190ad3c), cleanHead optimisation (97f1b76b4b), restoring the older adjust call order because it measured faster (77d9cb8937), timer race (fae6eb5f2f), heap sift simplification (d0051be847), debug tracing (38723f2b38). Left open: Stop/Reset racing a running timer returned wrong results.
• Go 1.24 — Done: Stop/Reset racing with a running timer now returns true when the run is cancelled (2ebaff4890); isSending bit cleared before locking the channel (ce60f70374), not used for tickers (48849e0866), then turned into an atomic.Int32 counter so more than 8 concurrent firings can be tracked (6a49f81edc; timer struct grows 96→104 bytes within its size class); avoid panic reading an expired synctest timer channel (4ce116a884). Left open: nothing noted.
• Go 1.25 — Done: bubbled timers behave like unbubbled ones: channel timers enter the bubble heap only when blocked on, and due timers run immediately rather than being queued, random order for same-instant timers (3432c68467, 3bd0eab96f); fake clock excludes monotonic time (5916bc5b57). Left open: nothing noted.
• Go 1.26 — Done: Timer.Reset(0) inside a synctest bubble no longer deadlocks on sendLock being taken twice (385dc33250). Left open: nothing noted.
• Go 1.27 — Done: user-visible asynctimerchan GODEBUG removed, deleting the async code paths (a5bc606a68); stale GODEBUG detection at startup (f5eb66f695). Left open: nothing noted.
• Go 1.28 — Done: user-visible time.Timer/Ticker copy detection (init bool replaced by self-pointer; f6ddbafcf0); delivered-time delta recomputed from a fresh clock just before send (277635b979); channel polling avoids duplicate timer checks and needless clock reads (713b754c97, 737d626d12); timers.verify stale check removed (eab6938192). Left open: nothing noted.

Runtime mutex: tri-state futex/sema locks → “spinbit” mutex

Span: Go 1.23 → Go 1.25 · Status at HEAD: only the spinbit implementation exists (all platforms except wasm); sched.lock is exempt from some behaviours

Paths not taken: Deeper-sleep futex mutex (superseded)

Earlier history: Semaphores, runtime locks and Notes (Go 1.0 → Go 1.17)

Design. The old lock2 was tri-state (unlocked/locked/sleeping) with separate futex and semaphore implementations; with many threads contending for a short critical section every unlock2 saw “sleeping” and woke another thread, which then joined the spin loop on the same cache line (throughput collapse). The final design (fd050b3c6d) unifies futex and sema platforms behind a semaphore abstraction (futexsleep/wakeup implemented as sema ops, 18c2461af3) and adds a “spinning” bit to the mutex word: when a waiter is already awake and spinning, other lockers sleep immediately and unlock2 skips the wakeup syscall. Sleeping Ms form a LIFO stack linked through the word; to bound starvation the bottom of the stack is periodically woken to spin. sched.lock is exempt from most of the new behaviour because Ms use it in sequence for thread-specific work and want low-latency handoff. The most efficient layout needs atomic.Xchg8, so that intrinsic was added for amd64, 386, arm, riscv64, mips64x, mipsx, s390x, wasm (841bb62f19, a8e2ecc8b1, 060bd25310, 58083b57d4, 266b0cff18, e15d14873f, 3c3b8dd4f0). Benchmarks in the commit: geomean 74.3ns→30.3ns (-59%) on one contention microbenchmark and -14.8% on another (13th-gen i7).

Timeline.

• Go 1.23 — Done: first attempt at futex lock redesign for profiling blame: share the sema lock2 structure with futex locks and double-link the waiting-M list (be0b569caa, d881ed6384, dfb7073bb8), plus unlock-time contention profiling (ba1c5b2c45, 8ab131fb12). All five commits reverted on 2024-05-29 (1be701a344, 5dead59add, afbbc2894b, 3f4be127bc, 9114c51521) because the patch series reduced the maximum throughput of contended runtime.mutex and regressed applications where that is the bottleneck. Left open: both a scalable lock and a correct blame-the-unlocker profile.
• Go 1.24 — Done: groundwork: futex OSes can use the sema-based mutex via new spinbitmutex GOEXPERIMENT value, off by default (18c2461af3); then the spinbit algorithm itself, gated by that experiment and enabled by default where Xchg8 exists (fd050b3c6d); control-flow cleanup (adc9c45587, -2.9% MutexCapture geomean). Left open: platforms needing emulated Xchg8 still on the old tri-state code; two implementations to maintain.
• Go 1.25 — Done: committed to spinbit everywhere including emulated-Xchg8 platforms, tri-state implementations deleted; wasm excepted (single thread) (7a177114df). Left open: nothing noted.

Contended runtime-lock profiling (mutex profile for runtime-internal locks)

Span: Go 1.22 → Go 1.27 · Status at HEAD: runtime locks appear in the mutex profile blaming the unlocker; call stacks gated behind runtimecontentionstacks

Design. Samples for contended runtime.mutex values are stored one-deep in the M (call stack + cycles waited) until it is safe to insert into the mprof bucket table (no runtime mutex held, not in a gscan state), because inserting itself takes the profInsert lock; unrecordable samples are folded into a sentinel _LostContendedRuntimeLock frame. In 1.25 the semantics were made to match sync.Mutex: unlock2 decides up front whether to sample; if so it uses a slower unlock path, releases the lock, walks the O(N) stack of waiting Ms to total the delay its critical section caused, and reports that with the unlocker’s stack (4d0f955a5e). Separately sync.Mutex profile semantic changed in 1.22 to count every blocked goroutine, matching Abseil (100 goroutines blocked 10ms = 1000ms; 1c00354013).

Timeline.

• Go 1.20 — Done: user-visible /sync/mutex/wait/total:seconds runtime metric (63ceff95fa); wait reason for goroutine blocked on a mutex made specific (a2c396ce00, 686b38b5b2); mprof locks split (1.19, 89c0dd829f^1.19). Left open: runtime-internal locks invisible in the profile.
• Go 1.22 — Done: user-visible mutex profile counts every blocked goroutine (1c00354013); runtime-internal lock contention profiled (0b31a46f1f, reverted 468bc94188, relanded 450ecbe905); GODEBUG=profileruntimelocks renamed runtimecontentionstacks and sentinel _LostContendedRuntimeLock (98fd8f5768). Left open: attributes the waiter’s own wait time, not the delay the critical section caused; futex locks “too late in the cycle” to fix.
• Go 1.23 — Done: acquireLockRank always acquires the M so m.locks is consistent when deciding whether profiling is safe (4bb67bc21e). Unlock-blame attempt reverted (see spinbit thread). Left open: blame semantics.
• Go 1.25 — Done: unlocker blamed (4d0f955a5e); fix for cputicks overflow causing zero-duration events (ac645eaa0e). Left open: nothing noted.
• Go 1.27 — Done: m.locks bit-split so lock/unlock count in steps of 16 while acquirem stays +1, so a mutex-count of zero is detectable when acquirem/releasem is interleaved (b99b8feaae; fixes dropped samples in TestRuntimeLockMetricsAndProfile); gscan vs profInsert rank deconflicted (29c1c946aa). Left open: nothing noted.

Scheduler core: spinning Ms, work conservation, stealing, STW interaction

Span: Go 1.18 → Go 1.28 · Status at HEAD: ongoing tightening of races in P handoff, STW and safe-point handling

Earlier history: The G-M-P scheduler: from global-lock M:N to per-P run queues, spinning Ms and work stealing (Go 1.0 → Go 1.17) · Channels and select (Go 1.0 → Go 1.17) · Stop-the-world, safe points and GC–scheduler interaction (Go 1.0 → Go 1.17)

Design. Standard G/M/P: per-P local runq plus runnext, global runq, spinning Ms (sched.nmspinning) that steal from random victims, idle P list (pidleget), idle M list. Invariant being defended across these commits is work conservation: when work is submitted and an idle P exists, a spinning M must exist or be created. Racing windows (“delicate dance” where a spinning M drops its P, rechecks queues, parks) produce lost wakeups.

Timeline.

• Go 1.18 — Done: redundant loop removed in runqget (3920d6f208); tracking time for scheduler latency only when needed (29ec902efc); sysmon NetBSD workaround skipped on >=9.2 (6406227d71). Left open: nothing noted.
• Go 1.19 — Done: work-stealing order randomness fixed: with GOMAXPROCS=12 only 4 coprime strides existed so position and stride were correlated; now chosen independently (510ad4561f). Scheduling decisions from schedule moved into findrunnable so any M in findrunnable can become a GC worker or trace reader and handoffp sees all work sources (fixes #44313, #39004) (13b18ec947). runqgrab yields instead of sleeping on OpenBSD/NetBSD (226346bb76, b93ceefa7b). Goroutine-profile STW reduced to fixed bookkeeping with a barrier in the scheduler (52bd1c4d6c). Left open: lost-wakeup race when a non-spinning M drops its P.
• Go 1.20 — Done: sched.needspinning: if work submission (wakep, injectglist) finds no idle P it sets the flag, and a non-spinning M about to drop its P rechecks it and becomes spinning instead; fixes a worst-case deadlock when injectglist finds nothing as all Ps go idle (8cb350d69a). A long list of schedt fields converted to atomic types (see sync/atomic thread). goschedIfBusy yields if gp.preempt (50664c236f). bgsweep “low-priority” heuristic: don’t yield if idle Ps exist, sweep >=10 spans between yields, cutting trace events by 90% (8451529e9a). Left open: nothing noted.
• Go 1.21 — Done: stale now used by netpolling Ms fixed (6f13d0bfe4); checkdead double-lock/startm handling (f229787aff, ff059add10); extra-M API cleanup and exclusion from SetMaxThreads (77b1f23af7, 734b26d4b9). Left open: nothing noted.
• Go 1.22 — Done: gosched calls wakep after pushing onto the global runq: without it, a spinning M that had just checked the global runq could park while the yielder’s G was stranded (loss of work conservation, with sysmon’s 10ms preemption the only backstop) (ad943066f6); global runq checked in the delicate dance (dd5db4df56); forEachP wrapper moves caller to _Gwaiting like stopTheWorld to avoid deadlocks with GC stack scanning (ff7cf2d4cd); m.locks-related and wirep throws moved to systemstack (37aed23f51). Left open: sysmon still preempts regardless of idle Ps (#60693 referenced).
• Go 1.23 — Done: idle-P race in injectglist (24070cf747); runnext disabled on wasm (no sysmon to prevent runnext ping-pong starvation) — added and reverted (0c64be46ac, a6a5c30d2b, 4afb155cdf); trace acquire/release across all P steals (c9d88ea2aa). Left open: nothing noted.
• Go 1.25 — Done: only one P at a time polls the network in findRunnable (352dd2d932), reverted for a CockroachDB benchmark regression (489917fc40), reintroduced (1e436ba668); explicit exclusion of a theoretical globrunqget->runqput->runqputslow recursion by refactoring runq/globrunq APIs (a42e337ede, +size field for gQueue/gList e0dba45c62); semrelease no direct G handoff on systemstack (a189516d3a); allpSnapshot stashed on M (740857f529). Left open: nothing noted.
• Go 1.26 — Done: P-to-M affinity across STW: sched.midle made doubly linked so a specific M can be pulled out, Ps restart on their previous M and STW preemption via goschedImpl keeps the G on its P (2263d4aabd, 4ebf295b0b; TestTraceSTW error rate 99.8% -> 1.9%); GC mark workers chosen in procresize during start-the-world to avoid sequential wakeps (a18aff8057); steal path avoids the 3us usleep before taking runnext when the victim is in a syscall, now that such Ps stay _Prunning (d55ecea9e5); debug mode extending _Grunning-without-P windows (7a8d0b5d53). Left open: “best effort” affinity; spinning M can still steal from a slow-restarting P.
• Go 1.28 — Done: spinning Ms don’t commit to a runnext steal if a STW became pending during the sleep (a22b5bf6c5, #81657); async-preemption signals sent outside sched.lock in forEachP/stopTheWorldWithSema, avoiding a pile-up of Ms sleeping on sched.lock (b5bbf5b660); worldsema asserted in forEachPInternal (14c1cd3e46); safe-point requests re-checked in execute (BenchmarkAllocation -8.6%) (5336092116); preemption retry delay starts after each attempt and missed retries when pausing goroutines fixed (480af5e85a, d4fc38b7b6). Left open: nothing noted.

Syscall path: eliminating _Psyscall, _Gdeadextra, and goroutine states

Span: Go 1.26 · Status at HEAD: shipped

Design. Previously a P in a syscall sat in _Psyscall and entersyscall/exitsyscall did an atomic store and CAS on the P status. _Psyscall was eliminated: the P stays _Prunning and syncronisation uses CAS on the G’s _Gsyscall status (7244e9221f, authored 2025-02, landed in the 1.26 cycle). The price is a newly possible combination—brief windows with a goroutine _Grunning but without a P—hidden from the execution tracer and flagged as risky in the message. Supporting pieces: _Gdeadextra status for goroutines on extra Ms (distinct from _Gdead since it may take a scan bit; df33c17091); no scheduling-latency tracking on _Grunning<->_Gsyscall (5b8e850340); direct optimistic CAS of atomicstatus in enter/exitsyscall (8683bb846d); nGsyscallNoP accounting fixes (745349712e, 81283ad339).

Timeline.

• Go 1.26 — Done: the above commits, plus user-visible runtime/metrics for goroutine scheduling states, total goroutines created and thread counts (13df972f68, ab8121a407, 78a3968c2c). Left open: “_Grunning-without-P” debugging only via an env debug mode (7a8d0b5d53).

GOMAXPROCS container awareness and dynamic updates

Span: Go 1.25 · Status at HEAD: default behaviour, under containermaxprocs/updatemaxprocs GODEBUGs

Earlier history: GOMAXPROCS, thread and resource limits (Go 1.0 → Go 1.17)

Design. internal/runtime/cgroup parses /proc/self/cgroup and mountinfo to find the leaf cgroup’s CPU quota/period (cgroup v1 and v2) using a single-scratch-buffer line reader (f12c66fbed, 06450a82b0). containermaxprocs (Linux) makes the default GOMAXPROCS the lower of the cgroup limit and the logical CPU count; updatemaxprocs (all OSes) makes sysmon periodically recompute the default and update GOMAXPROCS via a helper goroutine (updateMaxProcsG) if the CPU mask or cgroup limit changed (e6dacf91ff). Departure from the proposal: only the leaf cgroup is considered, with cached fds periodically reread; parent cgroup limits and migration are ignored, to avoid fd-leak-detection issues. Any explicit GOMAXPROCS env or call disables updates.

Timeline.

• Go 1.25 — Done: user-visible cgroup discovery and default-on behaviour (f12c66fbed, e6dacf91ff); rename to updateMaxProcsG (6160fa59b6); updatemaxprocs metric only when disabled (3b77085b40); guarantee of no GOMAXPROCS-update syscalls after a GOMAXPROCS call by synchronising with sysmon (for sandboxing) (e481a08e0e); custom GOMAXPROCS noted even when the value is unchanged (f069a82998). Left open: parent limits, cgroup migration (“we can consider changing this in the future”).
• Go 1.26 — Done: cgroup v1 non-root mount points handled, path validation and fuzz tests (c2af9f14b4, cead111a77, 67851547d8). Left open: nothing noted.
• Go 1.28 — Done: only the root of cgroup mounts is validated (f91a6faead). Left open: nothing noted.

testing/synctest bubbles

Span: Go 1.24 → Go 1.28 · Status at HEAD: GOEXPERIMENT removed in 1.26; all bubble violations are fatal; Test-style API

Design. A bubble is a tree of goroutines with a private fake clock. The runtime tracks bubble membership per g, marks goroutines blocked on channels, sync.Cond, sync.WaitGroup, timers, select{} as durably blocked; when every bubbled goroutine is durably blocked, the bubble’s clock jumps to the next timer; if no timer exists a deadlock panic is raised. Channels record the creating bubble (hchan.bubble), so cross-bubble operations are detectable. Timers inside a bubble live in a per-bubble heap and run when the bubble is idle. WaitGroups are associated with a bubble at first Add and only durably block Wait while associated (b170c7e94c), and are disassociated on Wait (9ae38be302).

Timeline.

• Go 1.24 — Done: internal synctest.Run/Wait with fake clock hooks in chan, coro, mgc, lockrank (d90ce588ea); user-visible experimental testing/synctest under GOEXPERIMENT=synctest (76e4efdc77); deadlock avoidance when copying stacks (1d28fa8c43); bubble membership propagated through syscall/js callbacks (a8eddaf758); Run return happens-after goroutine exits (3924fe92b6^1.25, 1.25). Left open: experiment only.
• Go 1.25 — Done: fake clock stops advancing once the main goroutine exits (d6c8bedc7b); no monotonic reading in bubbled time.Time (5916bc5b57); race-detector happens-before for AfterFunc (c0392e7e49); “bubble” terminology (ef7724c432); closing bubbled chans from outside panics (d596bc0e81); hchan.bubble pointer (0afcf9192c); WaitGroup association (b170c7e94c); cond-misuse fatal (21b7e60c6b); distinct deadlock message once the main goroutine is done (049a5e6036); lock rank for weak pointers vs synctest (b78e38065e); clearer bubbled stack traces (848a768ba7); timers aligned with unbubbled (3432c68467, 3bd0eab96f). Left open: some violations still recoverable panics.
• Go 1.26 — Done: all bubble violations are fatal panics (so e.g. context cannot be left inconsistent by a recoverable panic) (a8564bd412); synctest GOEXPERIMENT removed (4c0fd3a2b4); Reset(0) hang fix (385dc33250); specials lock (5dcedd6550). Left open: nothing noted.
• Go 1.27 — Done: concurrent race-context use in synctest timers avoided (784ea961a4); crypto/fips uses unbubbled monotonic clock (a81b20a431). Left open: nothing noted.
• Go 1.28 — Done: synctest.IsInBubble made inlinable when no bubble exists, since WaitGroup.Add calls it (ace5a7527c). Left open: follow-up CL not to use sync.Pool within bubbles referenced.

Coroutines (iter.Pull runtime support)

Span: Go 1.22 → Go 1.24 · Status at HEAD: stable runtime primitive (newcoro/coroswitch)

Paths not taken: Detecting coroutines in channel code (superseded)

Design. coro lets one goroutine hand its M/P directly to another without going through the scheduler queues: coroswitch marks the caller waiting, makes the target running, so iter.Pull costs roughly a function call plus switch rather than channel handoff. casgstatus is bypassed with an optimistic CAS fast path (reused later for syscalls, 8683bb846d).

Timeline.

• Go 1.22 — Done: iter, runtime: add coroutine support, exported under GOEXPERIMENT=rangefunc (a9c9cc07ac). Left open: interaction with thread-locked goroutines.
• Go 1.23 — Done: thread-lock state is bound to the coro at first newcoro; full lock state (counts and locked thread) must remain identical thereafter; simple nested/serial iter.Pull stays fast while exotic cases fail (dfe781e1eb); goexit propagation from iterator in Pull (9d2aeae72d). Left open: restrictions “may consider lifting”.
• Go 1.24 — Done: goroutine profile invariants upheld in coroswitch (2a98a1849f). Left open: nothing noted.

Asynchronous preemption: extended register state

Span: Go 1.18 → Go 1.28 · Status at HEAD: AVX2/AVX-512/Neon/LSX/LASX state saved and conservatively scanned

Earlier history: Preemption: cooperative stack-check preemption (1.2) to signal-based asynchronous preemption (1.14) (Go 1.0 → Go 1.17)

Design. mkpreempt.go generates per-arch asyncPreempt stubs. For vector code (simd) the preempted goroutine’s extended registers must be saved off-stack in a block (gp.xRegs) published after fully overwritten and scanned conservatively by GC so pointers living only in vector registers are kept alive.

Timeline.

• Go 1.18 — Done: restartable-sequence pcdata accepted in isAsyncSafePoint (8ff16c1990); maymorestack always-preempt and stack-moving hooks for testing (35c7234601, bd580a0d10). Left open: nothing noted.
• Go 1.19 — Done: loong64 asyncPreempt (740b0ebb76); arm64 uses stp/ldp (2e9facbdd4). Left open: nothing noted.
• Go 1.24 — Done: VZEROUPPER dropped from darwin/amd64 asyncPreempt (fe87b586c0). Left open: nothing noted.
• Go 1.25 — Done: arm64 resumes via return (77343fa646); internal/runtime explicitly excluded from async preempt (92a63bdfee). Left open: nothing noted.
• Go 1.26 — Done: scalar registers saved off-stack (af0c4fe2ca), AVX2/AVX-512 (55d961b202), arm64 Neon (955a5a0dc5), loong64 fcc/lasx/lsx (bd885401d5, 99cf4d671c), cherry-picked from dev.simd. Left open: GC scanning of that state.
• Go 1.27 — Done: conservative scan of extended register state (683aa8893a); loong64 fcsr0 (e76c910108). Left open: nothing noted.
• Go 1.28 — Done: AVX saved even without AVX2 (674806dda3); save blocks no longer zeroed (8f71bead49). Left open: nothing noted.

sync package API evolution

Span: Go 1.18 → Go 1.28 · Status at HEAD: stable; micro-optimisation continues

Paths not taken: Per-CPU sharded values: sync.Cache, distributed RWMutex, percpu.Sharded (still open)

Earlier history: sync.Mutex, RWMutex, WaitGroup, Cond, Once (Go 1.0 → Go 1.17) · sync.Pool (Go 1.3 → Go 1.17)

Design. The package is kept small: Mutex (now in internal/sync), RWMutex, WaitGroup, Cond, Once (+ OnceFunc/OnceValue/OnceValues), Pool, Map. Fast paths use atomics; hooks into runtime_Semacquire and notifyList.

Timeline.

• Go 1.18 — Done: user-visible Mutex.TryLock, RWMutex.TryLock/TryRLock (645d07819b), TryLock fix to acquire even when state != 0 (fdd67930a0); WaitGroup avoids a dynamic alignment check on 64-bit (ad7db1f90f); Map.Range re-entrancy doc (d5a5a13ad9). Left open: nothing noted.
• Go 1.19 — Done: happens-before documented for sync, atomic, finalizers (a71ca3dfbd); Cond guidance (d42a48828f); semaphore-treap scalability tests (7bad61554e). Left open: nothing noted.
• Go 1.20 — Done: user-visible Map.Swap, CompareAndSwap, CompareAndDelete (395323c4d0); Map and entry use atomic.Pointer (b0144b3843, 7366b199c5); WaitGroup state atomic.Uint64 (ee833ed72e); RWMutex counters atomic (e509452727); Once.done atomic conversion reverted (b53471a655). Left open: nothing noted.
• Go 1.21 — Done: user-visible OnceFunc, OnceValue, OnceValues (bb44c2b54e); Map memory model docs (f07910bd57). Left open: closure retention.
• Go 1.22 — Done: Once-wrapped functions are not kept alive after running (4f55a5af5e); Pool panics rather than throws on nil receiver (be0e0b06ac); Map.Range no longer always escapes (7015ed891c); Once uses atomic.Uint32 (be3d5fb6e6). Left open: nothing noted.
• Go 1.23 — Done: user-visible Map.Clear (5b6cd3d0cb, 3b0d39c5da). Left open: nothing noted.
• Go 1.24 — Done: sync.Mutex implementation moved to internal/sync so sync can later depend on internal/concurrent (frame skip for mutex profiling adjusted) (6c66005285); explicit noCopy fields (53b2b64b64); HashTrieMap added to Map tests/benchmarks (50f1888814); RWMutex upgrade/downgrade doc (bc0406333f). Left open: Map not yet replaced by HashTrieMap in this log.
• Go 1.25 — Done: user-visible WaitGroup.Go (822031dffc), f must not panic (9921537a96); fewer allocations in OnceFunc family via single captured struct (c5ff26a7a0); funcs not kept alive after panic (ee0d03fab6); Once.done as atomic.Bool (95611c0eb4). Left open: panic handling in WaitGroup.Go.
• Go 1.26 — Done: WaitGroup.Go no longer calls Done when f panics (95a0e5adc1); race annotations kept when panicking (8f74f9daf4). Left open: nothing noted.
• Go 1.27 — Done: linkname replaced with import (97259c5d6b); Cond.Broadcast lock guidance (1225feb0da). Left open: nothing noted.
• Go 1.28 — Done: Pool.Get only clears the private slot when non-nil: PoolGetEmpty/P=10 23.1ns -> 2.6ns (-88.8%) by avoiding cache-line contention with stealing Ps (d461d8f704). Left open: “sync: don’t use Pool from within synctest bubbles” in progress (referenced by ace5a7527c).

sync/atomic and runtime atomics

Span: Go 1.18 → Go 1.28 · Status at HEAD: typed atomics standard; runtime-internal atomics under internal/runtime/atomic

Paths not taken: Splitting the runtime package with sockdrawer (superseded) · 64-bit field alignment on 32-bit platforms (dormant)

Earlier history: sync.Map, atomic.Value and sync/atomic (Go 1.0 → Go 1.17)

Design. Go 1.19 added typed atomic values (Int32, Uint64, Bool, Pointer[T] …) that inline to primitives, hide the raw word so every access must be atomic, and guarantee 64-bit alignment (ffe48e00ad). Go 1.20 converted the runtime wholesale to these types (a ~50-commit series: schedt.*, p.*, g.atomicstatus, netpollWaiters, timeHistogram …), dropping padding and alignment hacks (1aa5730f49, bcd1ac7120); inline-cost tweaks (9dd9174fa5, 638c9aad5f) keep methods inlinable, and conversions of Pointer[T] are disallowed (6bead8f77a). Go 1.23 added public And/Or (192d65e46b) with arch-native runtime versions.

Timeline.

• Go 1.19 — Done: user-visible typed atomics (ffe48e00ad). Left open: runtime still uses old API.
• Go 1.20 — Done: the runtime-wide conversion (b464708b46, dd8cb66d0b, 5d7d50111f, 58a1dabfd1, 1eeef5d5b4, 88ef50e662, 3fbcf05d40, 951d2c657d, fa0e3bffb4, 3f65ddbfd3, 301ca7513f …); a revert for ncgocall (ef8414101f); user-facing hints toward typed API (0bd4710ca6). Left open: nothing noted.
• Go 1.23 — Done: runtime atomics moved to internal/runtime/atomic (4c2b1e0feb); user-visible atomic.And/Or (192d65e46b). Left open: nothing noted.
• Go 1.24 — Done: intrinsics marked noescape (e30ce3c498); loong64 acquire loads (geomean -74%/-45%) (3214129a83); arm native And8/Or8 (c5d424bfca). Left open: nothing noted.
• Go 1.27 — Done: PPC64 Add/And/Or SQCST (f4d1304904); linknames simplified (0dcd7ab1b9). Left open: nothing noted.
• Go 1.28 — Done: riscv64 Xaddint64 return fix (3b98eddbcd). Left open: nothing noted.

Runtime lock ranking

Span: Go 1.18 → Go 1.27 · Status at HEAD: generated from DAG; includes rwmutex modelling

Earlier history: Runtime lock ranking (static lock order checking) (Go 1.15 → Go 1.17)

Design. mklockrank.go consumes a DAG in the internal/dag language and generates lockrank.go with the transitive-closure partial order (2b8a9a484f); previously a hand-maintained array not even transitively closed. GOEXPERIMENT=staticlockranking checks acquire order per M. In 1.22 rwmutex got per-instance ranks with proper modelling of reader paths (9b4b3e5acc, dcbe772469).

Timeline.

• Go 1.18 — Done: rank ordering fixes (62c6ff4296). Left open: manual order.
• Go 1.19 — Done: partial order edge for trace/wbufSpans/mheap (482669db62). Left open: nothing noted.
• Go 1.20 — Done: DAG generation, unused ranks deleted, trace locks moved to the leaf (2b8a9a484f, ddfd639408, f42dc0de74, cc8bac8835, 44ff9bff0c, c5be4ed7df). Left open: names of ranks lost.
• Go 1.21 — Done: raceFiniLock, pinner and mspanSpecial/gcBitsArenas edges (f00e947cdf, 78df00dd6f, 807cce479f). Left open: nothing noted.
• Go 1.22 — Done: wakeableSleep edges (5a6f1b35d4, 23c0d30244); rwmutex (9b4b3e5acc, dcbe772469). Left open: nothing noted.
• Go 1.23 — Done: acquireLockRankAndM (4bb67bc21e); lockWithRankMayAcquire nosplit (3959d54c0b); sched/allp < timers rule removed (854dea060e). Left open: nothing noted.
• Go 1.25–1.27 — Done: vgetrandom rank (7b4d065267), spanSPMC lock rank fix (052c192a4c, 1.26), lockRank made int64 to shrink lockRankStruct on 64-bit (98da1aba90, 1.27). Left open: nothing noted.

Netpoller

Span: Go 1.18 → Go 1.28 · Status at HEAD: epoll/kqueue/IOCP/wasip1 pollers with tagged pollDesc pointers

Earlier history: Network poller: from the netFD select loop to the runtime-integrated epoll/kqueue/IOCP poller (Go 1.1 → Go 1.17)

Design. pollDesc holds read/write waiter G pointers (rg, wg) and flags. Races fixed over time: in 1.18 closing/errors flags were read racily and relied on x86 store barriers; on arm64 this deadlocked net/http tests, so accesses were made atomic (1b072b3ed5, 17b2fb1b65). In 1.21 a fdseq sequence number is packed into a taggedPointer stored in the kernel event, so events for a closed fd whose pollDesc was reused are ignored (not on 32-bit) (d5514013b6, a1674f3ee3).

Timeline.

• Go 1.18 — Done: atomic accessors for pollDesc (1b072b3ed5); net poll race fixes (17b2fb1b65). Left open: stale events.
• Go 1.20 — Done: netpollWaiters, netpollWakeSig, netpollInited atomic (aab8d2b448, 901b9233e6 …). Left open: nothing noted.
• Go 1.21 — Done: fd sequence tags (d5514013b6); wasip1 netpoll through poll_oneoff (c5c2184538); windows filter by completion key (7830180dcd). Left open: 32-bit tag.
• Go 1.22 — Done: netpollWaiters adjusted after goroutines ready and in netpollunblock (f51c55bfc3, eaa8419a72). Left open: nothing noted.
• Go 1.23 — Done: Windows websocket handling moved out of the runtime (d0dc93c8e1); netpoll event sources identifiable on Windows (1de46564a7). Left open: nothing noted.
• Go 1.24 — Done: kqueue EVFILT_USER one-shot events save about one syscall per wakeup (21b5f2637a). Left open: nothing noted.
• Go 1.28 — Done: 32-bit eventfd type confusion fixed (b9ba134da6). Left open: nothing noted.

Goroutine lifecycle and identity

Span: Go 1.20 → Go 1.27 · Status at HEAD: goroutine leak profile on by default (experiment removed 1.27)

Earlier history: Goroutine and M lifecycle: caches, ids, LockOSThread, Goexit, cgo extra Ms, deadlock detection (Go 1.0 → Go 1.17)

Design. g.goid became uint64 with atomic goidgen (dd8cb66d0b, b464708b46); parent goroutine ID is recorded and printed in stack traces (“in goroutine N”) (51225f6fc6). Goroutine leak detection uses the GC itself: goroutines blocked on unreachable sync primitives are reported via a new profile (8c68a1c1ab, 94 files). New goroutines inherit DIT state without OS thread locking (af14f67911) and secret state (e26a373785).

Timeline.

• Go 1.20 — Done: goid uint64 (dd8cb66d0b). Left open: nothing noted.
• Go 1.21 — Done: user-visible parent goroutine ID in tracebacks (51225f6fc6). Left open: nothing noted.
• Go 1.26 — Done: user-visible goroutine leak profile behind GOEXPERIMENT=goroutineleakprofile (8c68a1c1ab); DIT inheritance (af14f67911). Left open: experiment only.
• Go 1.27 — Done: goroutineleakprofile enabled by default then experiment dropped (253aa2a12a, 16449179ec); secret state inheritance (e26a373785). Left open: nothing noted.

Fast random numbers for the scheduler

Span: Go 1.18 → Go 1.27 · Status at HEAD: cheaprand, wyrand-based

Earlier history: Fast random numbers for the scheduler (Go 1.0 → Go 1.17)

Timeline.

• Go 1.18 — Done: fastrand switched to wyrand (483533df9e); fastrandn instead of modulo (ecb2f231fa). Left open: nothing noted.
• Go 1.19 — Done: fastrand64 (ebe1435fbb). Left open: nothing noted.
• Go 1.20 — Done: loong64 wyrand (3a41094107). Left open: nothing noted.
• Go 1.27 — Done: 32-bit cheaprand via 64-bit multiply; cheaprand64 as 64-bit wyrand (27fcec4d8f). Left open: nothing noted.

Changes in this slice that are not part of a thread are listed on the release pages: 1.18 · 1.19 · 1.20 · 1.21 · 1.22 · 1.23 · 1.24 · 1.25 · 1.26 · 1.27 · 1.28