Data structures and type metadata
Thread map, Go 1.0 – 1.17
| Thread | 1.0 | 1.1 | 1.2 | 1.3 | 1.4 | 1.5 | 1.6 | 1.7 | 1.8 | 1.9 | 1.10 | 1.11 | 1.12 | 1.13 | 1.14 | 1.15 | 1.16 | 1.17 | Status at go1.17 (or at the end of its era slice) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Runtime data structures and type metadata (Go 1.0–1.17) | |||||||||||||||||||
| Hash map core: from the first hashmap.c to the 8-entry bucket design | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (array of 8-slot buckets, tophash bytes, chained overflow, incremental growth) → continues as Swiss-table maps (GOEXPERIMENT=swissmap → only implementation) (1.24+) and Classic map maintenance and maps package helpers (pre-swiss) (1.18+) | |
| Map growth, iteration order, overflow buckets and GC interaction | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as Classic map maintenance and maps package helpers (pre-swiss) (1.18+) | ||||||||
| Map fast paths and compiler/runtime specialization | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as Classic map maintenance and maps package helpers (pre-swiss) (1.18+) | ||||||||||
| Hash functions and algorithm tables: memhash, AES hash, fallback hash | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (AES hash where available, wyhash-based fallback; per-map seeds; hash functions generated per key type) → continues as Map hashing and hash functions (memhash, aeshash, fallback hash) (1.23+) | ||||||
| Compiler-generated equality and hash functions | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as Compiler-generated equality/hash functions and type descriptor emission (reflectdata) (1.18+) | ||||||||
| Runtime type descriptors: layout, shrinking and name encoding | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (rtype with tflag, nameOff/typeOff/textOff, varint names, uncommon type following the base) → continues as internal/abi type layout unification and type metadata (rtype → abi.Type) (1.21+) | |||
| Interface values, itabs and type assertions | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (compiler-emitted itabs, runtime itab hash table) → continues as Interface conversion: itabs, type switch and assertion caches (1.19+) | ||
| reflect redesign: from string parsing to Type/Value structs | ● | ● | ● | ● | ● | still the design at go1.17 for the Type/Value API (opaque Value {typ, ptr, flag}); implementation tuned in 1.18+ → continues as reflect implementation: performance, allocation and escape behaviour (1.18+) | |||||||||||||
| reflect dynamic types: ArrayOf, ChanOf, MapOf, SliceOf, FuncOf, StructOf | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (synthesized rtype structures with GC info built in reflect) → continues as reflect implementation: performance, allocation and escape behaviour (1.18+) | ||||||||||
| reflect.Call, MakeFunc, method values and argument-frame GC precision | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | replaced by the register-ABI implementation in 1.17 (see register ABI thread) → continues as Register ABI completion for reflect, memmove/memclr and bytealg across architectures (1.18+) | ||||||
| Register ABI support in reflect and runtime assembly (Go 1.17) | ● | reflect assumes the register ABI only under GOEXPERIMENT=regabiargs and uses global register counts; not yet default → continues as Register ABI completion for reflect, memmove/memclr and bytealg across architectures (1.18+) | |||||||||||||||||
| reflect performance, caches and API additions | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as reflect implementation: performance, allocation and escape behaviour (1.18+) | |||||
| memmove, memclr, duff routines and pointer-write atomicity | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as memmove / memclr / memequal / internal/bytealg assembly optimisation (1.18+) | |||
| internal/bytealg and string/byte comparison primitives | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as memmove / memclr / memequal / internal/bytealg assembly optimisation (1.18+) | |||||
| Slices and append: growslice, makeslice and string/slice conversions | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (2x below 1024 old cap threshold, 1.25x above) → continues as Slice growth, string building and allocation-free bytes/strings helpers (1.18+) | |||||
| Strings: concatenation, conversion and printing helpers | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 → continues as Slice growth, string building and allocation-free bytes/strings helpers (1.18+) | ||||||
| Soft-float, 64-bit division and complex arithmetic helpers | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | still the design at go1.17 (Go implementations in softfloat64.go, vlop, complex.go) → no continuation noted | |||||||
| C-to-Go conversion of runtime data-structure code | ● | ● | ● | completed in Go 1.5; the runtime is Go with assembly stubs → no continuation noted | |||||||||||||||
| internal/reflectlite and checkptr | ● | ● | ● | ● | ● | still the design at go1.17 | |||||||||||||
Thread map, Go 1.18 – 1.28
| Thread | 1.18 | 1.19 | 1.20 | 1.21 | 1.22 | 1.23 | 1.24 | 1.25 | 1.26 | 1.27 | 1.28 | Status at HEAD |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Runtime: maps, built-in data structures, hashing, randomness and type metadata (Go 1.18–1.28) | ||||||||||||
| Swiss-table maps (GOEXPERIMENT=swissmap → only implementation) | ● | ● | ● | ● | ● | only map implementation since the experiment was deleted in 1.26 (2ae059ccaf); GOEXPERIMENT=mapsplitgroup (KKKKVVVV layout) still an opt-in experiment; micro-tuning ongoing in 1.27/1.28 | ||||||
| Map hashing and hash functions (memhash, aeshash, fallback hash) | ● | ● | ● | ● | hashing code for maps lives in internal/runtime/maps (moved from runtime), hybrid wyhash/AES keyed by MinAeshashSize, interface hash folds the dynamic type | |||||||
| Compiler-generated equality/hash functions and type descriptor emission (reflectdata) | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | one equality/hash function per "signature"; large types call subroutines; alg and noalg descriptors share one symbol | |
| internal/abi type layout unification and type metadata (rtype → abi.Type) | ● | ● | ● | ● | ● | ● | abi.Type is the single type-descriptor definition used by compiler, linker, runtime, reflect and reflectlite; direct/indirect interface flag lives in TFlag | |||||
| Interface conversion: itabs, type switch and assertion caches | ● | ● | ● | per-call-site caches in front of interface type switches/assertions; itabs generated via rttype | ||||||||
| reflect implementation: performance, allocation and escape behaviour | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | reflect.Value is stack-allocatable, many accessors inlineable, method Values eagerly materialised |
| Register ABI completion for reflect, memmove/memclr and bytealg across architectures | ● | ● | ● | register ABI unconditional on all ports; non-register fallback paths deleted per port | ||||||||
| memmove / memclr / memequal / internal/bytealg assembly optimisation | ● | ● | ● | ● | ● | ● | ● | ● | ● | ● | per-architecture hand tuning continuing; arm64 SIMD memmove landed then reverted in 1.28 cycle | |
| Slice growth, string building and allocation-free bytes/strings helpers | ● | ● | ● | ● | ● | ● | smoothed growth formula in nextslicecap, bytealg.MakeNoZero used by bytes/strings | |||||
| Runtime randomness: fastrand, chacha8rand and math/rand/v2 | ● | ● | ● | ● | ● | internal/chacha8rand generator with SIMD block functions on amd64, arm64, loong64, riscv64 | ||||||
| Concurrent hash-trie map (internal/sync.HashTrieMap) | ● | ● | stable internal API used by unique (and later sync.Map) | |||||||||
| iter.Pull and range-over-func iterator types | ● | ● | ● | stable; documentation churn | ||||||||
| Classic map maintenance and maps package helpers (pre-swiss) | ● | ● | ● | ● | old implementation deleted in 1.26 | |||||||
Runtime data structures and type metadata (Go 1.0–1.17)
Reader: Claude Sonnet. Cites 1,028 of 1,319 commits in this slice; every non-noise commit is cited, plus 13 larger commits the noise filter had set aside, which a separate gap reader described. Written in one pass from the design-heavy messages and subject lines; the long tail is cited in grouped “Other” items.
Release digest for this slice
- Go 1.0 — Four years of foundations: hashmap.c, the C
ifacecode, the 2009 reflect rewrite from string parsing to compiler-laid-out type structs, the 2011 Type/Value redesign and opaqueValue, per-map hash seeds and randomized iteration. - Go 1.1 — The 8-slot bucket hashmap with incremental growth and fast-path routines, AES hash, reflect.MakeFunc/Select/Convert/*Of and method values, the removal of run-time code generation, ARM softfloat conditional execution and faster ARM division.
- Go 1.2 — Hash lookup shortcuts, hashmap internals get typed GC info instead of bespoke GC code, itabs from persistentalloc, faster x86 memmove, Slice3, and ARM softfloat/preemption fixes.
- Go 1.3 — Map accessors take key/value by reference, evacuated bit moved into tophash, intra-bucket iteration offset, precise GC
reflect.Value(4 words), smarter slice growth, SSE memclr. - Go 1.4 — The C-to-Go conversion of maps, slices, hashing, interfaces and printing; the 2-bit GC type info; direct-interface policy moved into the compiler; random start bucket for iteration; 3-word
Value; reflect call-frame cache. - Go 1.5 — Pointer-free maps unscanned, overflow pointer at bucket end, new fallback hash, 1-bit pointer bitmaps in type data,
FuncOf/ArrayOf, duff routine generation and assembly cmpstring. - Go 1.6 — Atomic pointer writes in memmove/memclr, aeshash DoS hardening, removal of
type.zero, concurrent-map-misuse detection. - Go 1.7 — Descriptor shrinking (nameOff/typeOff, static name encoding, tflag, embedded rtype fields),
StructOf, compiler itabs and pointer-shapedconvT2I, growslice/makeslice division avoidance. - Go 1.8 — Same-size map growth to bound overflow buckets, duplicate-itab handling across shared libraries, typemap/typelinks work,
reflect.Swapper,decoderune. - Go 1.9 —
sync.Map-backed reflect caches,hmap.extraand overflow preallocation,MakeMapWithSize, the compiler’sdxxx-to-LSym refactoring and deterministic signature ordering, C99 complex division. - Go 1.10 — A large series of map micro-optimizations (evacuate refactor, fast-path copies, specialized memhash), the new open-addressed itab table with compile-time function pointers, MIPS soft-float.
- Go 1.11 —
internal/bytealgcreated, arm64 aeshash/memmove/Compare/Index, map fast routines in type-specific files, old ARM softfloat deleted, method-set sorting in the compiler. - Go 1.12 —
emptyOne/emptyRestearly exit in map scans, overflow-checked multiplication in make/growslice/makemap, growslice bulk barrier, reflect return-value scanning fix,MapRange. - Go 1.13 —
internal/reflectlite,reflect.IsZero,ifaceeqdirect fast path,kindNoPointersremoved, StructOf GC fixes, arm bytealg. - Go 1.14 — Hash functions generated only for map-key types with run-time interface hashing,
-d=checkptrsupport, riscv64 bytealg/reflect, atomic memmove on arm64. - Go 1.15 — Cheap-checks-first generated equality algorithms, small-integer interface conversions without allocation, typehash equal to compiler hashes, quiet NaNs in softfloat.
- Go 1.16 — Map seed rotation on clear, itab and GC-map content-addressable symbols, append growth keyed by capacity,
Zerobuffer, exported/non-exported interface method split,notinheappointers indirect. - Go 1.17 — Register ABI support in reflect and runtime assembly (
internal/abi,makeFuncStub/methodValueCall), wyhash 32-bit fallback, varint type names,VisibleFields, slice-to-array-pointer conversion,CanConvert.
Hash map core: from the first hashmap.c to the 8-entry bucket design
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (array of 8-slot buckets, tophash bytes, chained overflow, incremental growth) → continues as Swiss-table maps (GOEXPERIMENT=swissmap → only implementation) (1.24+) and Classic map maintenance and maps package helpers (pre-swiss) (1.18+)
Paths not taken: Hash-map alternatives before Swiss tables (superseded)
- talk Inside the Map Implementation. GopherCon 2016. Jul 12, 2016. listed in Go: A Documentary
- tracking issue #3885 runtime: profile and tune map code referenced by 2 of this thread's commits
Design. The first map implementation is src/runtime/hashmap.c (2008, “mike’s map code”, bc0b4f0d2a; an earlier map.c was removed by that commit). In 2008 keys of pointer type were allowed by treating them like ints (5adbacb8e7). The table was a chained hash table whose pieces were freed on resize in 2010 (21518ea2af); large values were supported by b198c8612e; the spec rule “a missing key yields the zero value” was implemented in the runtime in 2010 (e781303e27) and descriptive panics for writes to nil maps followed (f2852ba618). Range over a nil map and deleting from a nil map were made legal (86145611b0, 28a50c7f51).
The structure that shipped as the long-lived design is Keith Randall’s rewrite (00224a356a, Go 1.1): the hashtable is “an array of 8-entry buckets with chained overflow. Each bucket has 8 extra hash bits per key to provide quick lookup within a bucket. Table is grown incrementally.” The commit reported Go time dropping from 0.51s to 0.34s and added hashmap_fast.c (specialized lookups, see the fast-path thread) plus compiler changes in cmd/gc/walk.c. The Hmap header shrank to a 48-byte allocation in Go 1.2 (4042b77776). Several first-bucket and lookup shortcuts followed: lazy allocation of the first bucket array (5b3ff61be6, NewEmptyMap 196→107 ns, 2→1 allocs), avoiding string hashing for small single-bucket maps by comparing lengths first (45b54ee7fb, a696ae56db), and faster range over an empty map (7b9df09261).
In Go 1.3 the runtime/compiler interface changed so that key and value are passed to map accessors by reference rather than by value (3278dc158e), part of removing vararg C calls for exact stack scanning; a shared zero-memory area, allocated by the linker for all non-reflect map types, is returned on a miss. Go 1.4 converted the whole implementation from C to Go (0c6b55e76b, “a bit slower, but not painfully so”, with the hash/eq stubs and nosplit still on the wishlist) and moved constants from the map header into the map type (668a55a882). Go 1.5 moved the overflow pointer to the end of the bucket so that pointers to zero-sized keys/values cannot point past the bucket into the next object (fbc56cf050; the nacl/amd64p32 padding fix is c5dff7282e), skipped equal(k,k) for reflexive key types (df1739c77d), and stopped scanning maps whose keys and values contain no pointers by keeping all overflow buckets alive through a separate array (85e7bee19f). In Go 1.6 the zero-value pointer in the type descriptor stopped being used: a miss now returns a runtime-owned pointer (38519e69d0) and the zero field was removed (af78482d6b); before that, reflect-created map types had their zeros allocated on demand and shared (7e26a2d9a8). Go 1.11 removed the hmap field from map types (4ebc67d334) and Go 1.12 introduced emptyOne/emptyRest slot states (4b05c01f9c). Go 1.9–1.10 added hmap.extra, covered in the growth thread.
Timeline.
- Go 1.0 — Done:
- 2008–2009:
hashmap.c(bc0b4f0d2a), pointer keys (5adbacb8e7), hash alignment fix for data >= 8 bytes (9ed2960de8),forloops in hashmap (1f1ae404f8). - 2010–2011: zero value on missing key (
e781303e27), large values (b198c8612e,8599a83727), free old pieces during resize (21518ea2af), memory-leak fix (e7e5c5adb3), nil-map panics and nil range (f2852ba618,86145611b0),gofix -r mapdeletefor the delete builtin change (313c8224d5), randomized iteration start (e40d6e066a, user-visible iteration order is no longer predictable). - 2011-12 to 2012-03: algorithm in type data became a table pointer so compiler-generated code can be called, and the top-level map structure was trimmed from 104/80 to 24/12 bytes on 64/32-bit (
b9ccd077dc); per-map hash seeds fromfastrand1(85aeeadaec). Left open: seed was not yet cryptographically random (“Fastrand1 still needs to be updated”).
- 2008–2009:
- Go 1.1 — Done: user-visible 8-slot bucket hashmap with incremental growth (
00224a356a), lazy first bucket (5b3ff61be6), string-length shortcut (45b54ee7fb), map reads made multithread-safe by doing grow work only on insert/delete (0e7144a875; an earlier “grow work during reads”07b6add0ca1.2 was reverted in 1.2 by7f0ee023ba1.2), atomic flag updates (b6bfc92df3), seed always mixed at the start of hashing (63bee953a2), delete on nil map is a no-op (28a50c7f51), precise GC of hashmaps throughhash_gciter(1e01fba2fc), collision detection debug (bca01cd0bf). Left open: GC treatment of map internals was bespoke and later removed. - Go 1.2 — Done:
Hmapcut to 48 bytes (4042b77776), hash-lookup shortcuts for same-length strings (a696ae56db), type information recorded for the internal structures and all hashtable-specific GC code removed (fb376021be, preceded by0df438c683telling GC not to scan them), fix for iterator/growth/NaN-key interaction (869368a528), shorter lookup stack frames (a3f842a4c1). Left open: accessors still took key/value by value via C varargs. - Go 1.3 — Done: key/value by reference plus shared zero area (
3278dc158e, race-detector follow-upsc0f2294577,1cc2ff8fc7), bucket size computed before building bucket type (375b7bb767), shorter bucket type name (20f99ffa3e), fullHiterinitialization (fc6befba3c), reflect-created map types given a correct bucket type (5bc1cef869: previouslymap[unsafe.Pointer]unsafe.Pointer, GC overscanned up to 120 bytes formap[int8]struct{}). - Go 1.4 — Done: map implementation converted to Go (
0c6b55e76b), constants moved from header to map type (668a55a882),hmaptype size verified (fe3ee5741d), pass correct size to malloc for the new GC (7bcbdbd904). Left open: “room for improvement (saving space so we can use nosplit, and removing the requirement for hash/eq stubs)”. - Go 1.5 — Done: overflow pointer at end of bucket (
fbc56cf050,c6669e7af5,c5dff7282e), reflexive-key shortcut (df1739c77d), pointer-free maps not scanned (85e7bee19f,52dadc1f31noscan fix), typed memmove with write barriers for map contents (54bb4dc390), reflect map zeros on demand (7e26a2d9a8,d820d5f3ab). - Go 1.6 — Done: stop returning
t.zeroon a miss (38519e69d0,af78482d6b), skip key overwrite on update when not needed (00c638d243), best-effort detection of concurrent map misuse via a flag next toh.count(50c5042047). - Go 1.7 — Done: no algs generated for map bucket types (
a6abc1cd70), no write barrier on iteration (974c201f74). Left open: nothing noted. - Go 1.8 — Done: overflow-bucket limit (see growth thread,
9980b70cb4), iterator concurrent-map check fix (e492d9f018),bmap.tophashdocumented (bf776a988b). - Go 1.9 — Done:
hmap.extra,newoverflow,bmap.setoverflow, preallocated overflow buckets (a41b1d5052,619af17205,17d497feaa,94e44a9c8e), 64-bitoverLoadFactor(b666f2860b), bad size hint no longer panics (f9531448b8),reflect.MakeMapWithSize(6c5a819a5e),mapassign_*usetypedmemmovefor keys (579120323f), marking “writing” delayed until after the first alg call (04fc887761). - Go 1.10 — Done: the large map refactoring series by Josh Bleecher Snyder and Martin Möhrmann, described in the fast-path and growth threads (starting with
64bd2c49b4,445717530c,aca92f352d,cbc4e5d9c4; see those threads for the rest). - Go 1.11 — Done: fast routines moved to type-specific files (
d58593d8aa),hmapfield removed from map types (4ebc67d334),new(hmap)in makemap (bf9f1c1503), map delete always clears the value because compound-assignment optimizations rely on it, at +12–17% onMapDeletefor non-reference types (b080abf656), literal-key string allocation avoided form[T{string(b)}](a0f57c3fd01.12). - Go 1.12 — Done:
emptyOne/emptyRestmarkers let bucket scans stop early (4b05c01f9c,MegOneMap8.91→5.53 ns), delete updates entries after the new last element (df2bb9817b), multiplication-with-overflow-check inmakemap(e85b8db604), panic on uncomparable key type even for empty maps without growing the header (ed15e82413). - Go 1.13 — Done: “key”/“element” terminology applied across the implementation (
73cb9a1cb3). - Go 1.14 — Done: hash functions generated only for key types, moved into the map type (
36f30ba289, see hashing thread); NaCl overflow padding removed (ba97d0d84d). - Go 1.16 — Done: hash seed rotated when a map is cleared (
93810ac1f4,1a119edd8b), nomemclrcall for keys inmapdelete_fast*(f979d072d3). - Go 1.17 — Done: bucket type descriptor loosened (
15085f8974),mapassign_faststroverwrites existing keys (23ffb5b9ae), pointer arithmetic syntax simplified in mapaccess (0da9eff503). Left open: nothing noted; the 8-slot bucket stays until Swiss tables.
Map growth, iteration order, overflow buckets and GC interaction
Span: Go 1.0 → Go 1.12 · Status at go1.17: still the design at go1.17 → continues as Classic map maintenance and maps package helpers (pre-swiss) (1.18+)
Design. Growth in the 8-slot design is incremental: when the load factor is exceeded (initially computed with floating point, later with overLoadFactor in integer math: 733567a186, 64-bit calculation b666f2860b, load-factor bug fix dbe3522c7f), hashGrow allocates a bucket array twice as large and each insert or delete evacuates the old bucket it touches plus one more from an advancing mark. Reads in Go 1.1 first tried to do grow work too (07b6add0ca) but this broke freebsd-386 and was reverted (7f0ee023ba); the final rule is that only writes evacuate (0e7144a875). The “evacuated” state was originally stored in the low bit of the overflow pointer; because the GC wants correctly typed, aligned pointers it was moved into the tophash entries (9aae6c1a8b).
Iteration order has been deliberately randomized since 2011 (e40d6e066a). In Go 1.3 the randomization changed from “start at a random bucket” to “start at the first bucket but walk each bucket at a random offset” (3be4d95731), which made single-bucket maps (<= 8 entries) iterate in one of only 8 orders; Go 1.4 returned to a random starting bucket for larger maps, bringing the iterate-and-delete pattern from O(n^2) to O(n log n) (55c458e05f, MapPop10000 270 ms to 6 ms) and kept intra-bucket randomness always (251daf8650).
Overflow handling got three mechanisms. First, Go 1.8 bounded overflow-bucket growth (9980b70cb4): after repeated fill-and-delete (issue 16070) overflow buckets were never reclaimed. Instead of repacking (which disturbs iterators) the runtime reuses the existing growth path with a same-size grow (“sameSizeGrow”) once enough overflow buckets have accumulated; the counter is 16 bits and incremented stochastically for large maps so the header does not grow. Second, Go 1.9 turned the pointer-free-map overflow slice into a general hmap.extra (a41b1d5052), added newoverflow and preallocated overflow buckets based on a simulation of inserting the midpoint element count for each bucket-array size (94e44a9c8e). Third, evacuation was made more consistent: the evacuation mark skips buckets already evacuated (064e44f218), and in Go 1.10 the evacuate routine was unified for X/Y destinations (77a9cb9b4c, removing a data-dependent branch said to be mispredicted about half the time), split into advanceEvacuationMark (e0b34e7be7), made an evacDst top-level type (776c33ee5e) and given specialized evacuate_fast32/64/str copies (4cf7f909c6, 3c182a12f7, db9f5917fc, 0480bfe57f, 8adaf68570) that drop handling of non-reflexive and indirect keys. Pointer-free bucket memory is no longer cleared on grow (3f972df4a7) and delete clears only pointer-containing memory (0c7fd56951). hmap.extra.overflow was split into overflow and oldoverflow fields (1d3ad6733e).
GC interaction passed through three stages: bespoke hashmap GC iterator code (Go 1.1, 1e01fba2fc), compiler/reflect-generated type information for the bucket type with that code deleted (Go 1.2, fb376021be), and in Go 1.5 skipping scan when neither key nor value has pointers (85e7bee19f). Reflect’s MapOf had to build matching bucket types (3660b532ac, 5bc1cef869).
Timeline.
- Go 1.0 — Done: randomized iteration start (
e40d6e066a) and incremental-growth predecessor “free old pieces” (21518ea2af). Left open: iteration over one bucket still deterministic. - Go 1.1 — Done: incremental growth with 8-slot buckets (
00224a356a), no grow work in reads (0e7144a875), GC held off during insertion (54dffda2b6), maps freed more aggressively (bcb39a778c), precise GC (1e01fba2fc). Left open: evacuated bit in pointer’s low bit. - Go 1.2 — Done: typed GC data replaces bespoke GC code (
fb376021be), iterator/NaN/growth bug (869368a528). Left open: nothing noted. - Go 1.3 — Done: evacuated bit moved into tophash (
9aae6c1a8b), intra-bucket iteration offset (3be4d95731,8454e2c287). - Go 1.4 — Done: random starting bucket for bigger maps (
55c458e05f,251daf8650). - Go 1.5 — Done: overflow slice for pointer-free maps (
85e7bee19f), pointer bucket layout (fbc56cf050). - Go 1.8 — Done: same-size grow to bound overflow buckets (
9980b70cb4). Left open: counter is approximate for large maps by design. - Go 1.9 — Done:
hmap.extra,newoverflow, preallocation, consistent evacuation, size-hint fixes (a41b1d5052,619af17205,94e44a9c8e,064e44f218,f9531448b8). Left open: commit message notes follow-ups such as reusing freed overflow buckets (issue 19992) and size-hint-based estimates (19931). - Go 1.10 — Done: evacuate refactors and fast-path copies (list above), tophash helper (
445717530c),makemapdynamic checks moved to the compiler (54125d191d,bb31217579forismapkey),makemapint hint calls (cbc4e5d9c4), fixedmakemap64signature (15d5fa0291), hiter zeroing avoided (034d825ea3,b6296426a0), mask shifts on x86 (f7aa454c58), NaN evacuation simplified (8a9d4184e6). - Go 1.12 — Done: early exit on
emptyRest(4b05c01f9c), delete re-marks trailing empties (df2bb9817b),bucketCntbounded scanning. Left open: delete used the “safe but non-optimal” choice of emptyOne in some cases (message of4b05c01f9c).
Map fast paths and compiler/runtime specialization
Span: Go 1.1 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Classic map maintenance and maps package helpers (pre-swiss) (1.18+)
Design. hashmap_fast.c appeared with the new hashmap (00224a356a) to give int32/int64/string keys dedicated access routines without calling the generic hash/eq through the type; the compiler (walk.c) selects them. The Go 1.4 port kept them as mapaccess1_fast32/64/str and friends (0c6b55e76b). The big optimization session is Go 1.10: strength-reduced key pointer arithmetic using add (dad5d76e8f, MapStringKeysEight_* ~-40%; 61043d4671, 29e6bdc69c, 83ae9b07b8), checking length before tophash in string maps, preferring direct key comparison over tophash when comparison is cheap (ec0e2edd3b), refactoring insertion-slot tracking so b.tophash does not escape (8585f9fdb1), storing new keys in mapassign_fast32/64 with direct assignment or writebarrierptr instead of typedmemmove after an earlier last-minute 1.9 fix (25bbb695c8), and only clearing the key’s pointer in mapdelete_faststr (dc10aeeff8). memhash32/memhash64 were specialized (2a56b023af, 176cd48e57 for big endian). A walking helper removed a nil check in the inner loop (0b0cc4154e, 77871cc664, c50a9718a6). Go 1.11 split the fast routines into map_fast32.go, map_fast64.go, map_faststr.go (d58593d8aa). Compiler-side specializations include avoiding string allocation for m[string(b)] and its struct/array-literal generalization (a0f57c3fd0), and make(map) call forms with int hints (cbc4e5d9c4).
Timeline.
- Go 1.1 — Done: fast access for common key types (
00224a356a), string-map small-table shortcuts (45b54ee7fb). - Go 1.2 — Done: lookups optimized with same-length string checks (
a696ae56db,BenchmarkSameLengthMap43→4 ns). - Go 1.4 — Done: fast routines in Go (
0c6b55e76b). - Go 1.7 — Done:
appendfast-path spill avoided (6b33b0e98e, slice side). - Go 1.10 — Done: the series listed above (
dad5d76e8f,61043d4671,29e6bdc69c,0c7fd56951,ec0e2edd3b,8585f9fdb1,25bbb695c8,2a56b023af,dc10aeeff8,aca92f352d,dabc361847). Left open: benchmarking was “impeded by #21546” per several messages. - Go 1.11 — Done: type-specific files (
d58593d8aa). - Go 1.16 — Done: no
memclrcall for keys inmapdelete_fast*(f979d072d3; Pointer deletes -16..-30%). - Go 1.17 — Done:
mapassign_faststrfixed to overwrite existing keys (23ffb5b9ae).
Hash functions and algorithm tables: memhash, AES hash, fallback hash
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (AES hash where available, wyhash-based fallback; per-map seeds; hash functions generated per key type) → continues as Map hashing and hash functions (memhash, aeshash, fallback hash) (1.23+)
- discussion Design discuss: Switch hashmap implementation to SipHash-2-4 golang-dev
- motivating issue #25237 runtime: opportunistically rotate map key seed referenced by 2 of this thread's commits
- motivating issue #2630 runtime: hash function should be randomized to prevent DOS attacks referenced by 1 of this thread's commits
Design. Early runtime hashing was an alg table per type: Go 1.0 replaced a uint8 algorithm id with a table pointer (b9ccd077dc), with hash and equal functions taking a result pointer so compiler-generated code could be called from C. A float64 hash bug was fixed twice in 2012 (022aac7883, facee93a86) and Go 1.1 improved continuity of the hash computation (51fe5444fa) and added the AES-NI hash, first for compound objects (6b57329268). Go 1.4 converted hashing to Go calling convention with real closures (a2a9768414, 7aa4e5ac5f for equality), made non-empty interface hashing use something other than return 0 (f098a29630) and hashed floats on bytes rather than words so maps of floats get linear performance (5fbcdb2661); aeshash was disabled at compile time on NaCl (18ab35f7ba).
The fallback hash (used on arm, power and x86 without AES) was replaced in Go 1.5 with “a mixture of ideas from cityhash and xxhash” (cda0ea1c0e: ~no slower on small objects and up to ~50% faster on large ones on ARM), seeded with startup randomness (ce5cb037d1), and the size argument was removed from hash and equal algorithms (d5e4c4061b; sizes needed by generic code come through a closure that points at an assembly stub, binary 0.07% smaller). Go 1.6 made aeshash DoS-resistant by scrambling the seed before xoring with input and using table-dependent seeds for all stripes of strings > 16 bytes, costing 4 aesenc instead of 3 (91059de095, small hashes +5–10%). Go 1.10 specialized memhash32/64 (2a56b023af), Go 1.11 implemented aeshash on arm64 (73e0c302b2, Hash64 196→73 ns), and Go 1.17 switched the 32-bit fallback to wyhash (78f9015236, Hash64 121→102 ns).
Hash-function generation changed in Go 1.14: instead of generating hash functions for every type “just in case”, only map key types get them, and interface{} keys are hashed at run time by type dispatch (36f30ba289). The hash function moved from the key type to the map type, leaving a single equal in the descriptor (cmd/go dropped from 504 to 10 generated hash functions, -1.0% size, interface-key maps ~20% slower). Go 1.15 added a pointer fast path for interface hashing (afd691c579, MapInterfacePtr -33%) and made runtime typehash match compiler-generated hashes exactly because a reflect-made map assigned to a named map type could otherwise change hash function (2b8e60d464).
Seeding and DoS: per-map seeds (85aeeadaec), seed always incorporated first (63bee953a2), and in Go 1.16 the seed is rotated when a map is cleared via the for k := range m { delete(m,k) } idiom or by clearing (93810ac1f4, 1a119edd8b).
Timeline.
- Go 1.0 — Done: alg table pointer (
b9ccd077dc), float64 hash fixes (022aac7883,facee93a86), per-map seeds (85aeeadaec); 2009 alignment fix for data >= 8 (9ed2960de8). - Go 1.1 — Done: hash continuity (
51fe5444fa), always-first seed (63bee953a2), AES hash for compound objects (6b57329268), faster string equality in the runtime (77f3e189d2, identical-pointer case 51→7 ns;3d5daa2319, 4K equal 2187→120 ns on amd64). - Go 1.4 — Done: hash in Go (
a2a9768414,ea3ac6ba75), better interface/float hash (f098a29630,5fbcdb2661), equal in Go (7aa4e5ac5f), NaCl aeshash off (18ab35f7ba). - Go 1.5 — Done: new fallback hash (
cda0ea1c0e), randomized fallback seeds (ce5cb037d1), size argument removed (d5e4c4061b),goalgremoved (b1f29b2d44). - Go 1.6 — Done: aeshash DoS hardening (
91059de095). - Go 1.7 — Done: hash algs initialized before typemap so shared-library typemaps (themselves maps) work (
f135c32640); eq/hash generation simplified (199cc194ac). - Go 1.10 — Done: specialized memhash32/64 (
2a56b023af,176cd48e57), CPU-feature detection refactor for 386/amd64 (5a6c5809901.9 in 1.9). - Go 1.11 — Done: arm64 aeshash (
73e0c302b2). - Go 1.14 — Done: hash only for map keys, runtime interface hashing (
36f30ba289). - Go 1.15 — Done: interface pointer hashing fast path (
afd691c579),typehashequals compiler hash (2b8e60d464). - Go 1.16 — Done: seed rotation on clear (
93810ac1f4,1a119edd8b). - Go 1.17 — Done: wyhash fallback for 32-bit (
78f9015236). Left open: nothing noted.
Compiler-generated equality and hash functions
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Compiler-generated equality/hash functions and type descriptor emission (reflectdata) (1.18+)
Design. From Go 1.0’s alg table pointer (b9ccd077dc), struct and array types get compiler-synthesized eq and hash functions (cmd/gc/subr.c/alg.go, later reflectdata/alg.go). Go 1.7 simplified their generation (199cc194ac) and made alg prologue/epilogue match (e0032df81e), removed Type.Down uses (d3794f88e9), and stopped generating algs for map buckets (a6abc1cd70); alg generation after dowidth was fixed in 1.4 (8d47b08255). Go 1.9 sped up small-array comparison (3bdc2f3abf). Go 1.11 ensured alg functions are generated when the compiler calls them (c801232525).
Go 1.15 is the main optimization pass, by Josh Bleecher Snyder: order checks cheapest-first. Runtime calls (memequal, efaceeq) go last (79648bde2d); string equality is split into a cheap length half and a late contents half (daae72e88e, 5029c3671d); the same for interfaces (type pointer first, f4e13b83aa, 1cc7be89a9); arrays of interfaces and strings use two loops (7eab9506c9, 1c4e9b2eda, about 17% larger code) and small arrays unroll the first loop (c9591d73ef); an ordering bug was fixed (8c8045fd38); AlgKind became a stringer (ab3bd2c15f). Go 1.16 introduced a single not-equal label to avoid block recombination (c95bd2e6a9), and Go 1.17 reorganized the array hash loop (617383377f) and guaranteed no unaligned loads on strict-alignment architectures (873401df5b). Go 1.14 limited hash generation to map keys (36f30ba289) and Go 1.13 stopped marking the argument array as noalg (343b7fa626).
Timeline.
- Go 1.0 — Done: alg table (
b9ccd077dc). - Go 1.4 — Done: alg generated after dowidth (
8d47b08255). - Go 1.7 — Done: simplified generation, no bucket algs (
199cc194ac,e0032df81e,d3794f88e9,a6abc1cd70). - Go 1.9 — Done: cheaper small array compare (
3bdc2f3abf). - Go 1.11 — Done: alg functions always generated when called (
c801232525). - Go 1.13 — Done: argument array no longer noalg (
343b7fa626). - Go 1.14 — Done: generate hashes only for map keys (
36f30ba289), zero-value size constant (8e4399ff77). - Go 1.15 — Done: cheap-checks-first eq algs (
79648bde2d,daae72e88e,5029c3671d,f4e13b83aa,1cc7be89a9,7eab9506c9,1c4e9b2eda,c9591d73ef,8c8045fd38,ab3bd2c15f). - Go 1.16 — Done: shared not-equal label (
c95bd2e6a9). - Go 1.17 — Done: hash loop reorganized, unaligned-load guard (
617383377f,873401df5b),memequal_varlenreferenced as ABIInternal (411860251e).
Runtime type descriptors: layout, shrinking and name encoding
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (rtype with tflag, nameOff/typeOff/textOff, varint names, uncommon type following the base) → continues as internal/abi type layout unification and type metadata (rtype → abi.Type) (1.21+)
- tracking issue #6853 all: binaries too big and growing referenced by 13 of this thread's commits
Design. In 2008–2009 reflection parsed type-signature strings at run time (a cache avoided repeated parsing, 842e1a9aa7; tags and PtrTo needed special support 12a3435869). In July 2009 the compiler began laying out the data structures ahead of time and the reflect structures were unified with the interface runtime structures (4793400bd1, “no more parsing of strings”; a7b4e9f03e, 58c4142e74); the old parsing runtime Type was deleted in 2012 (6a75ece01c, 121a16fa95). Further rtype changes before Go 1.1: pointer word in CommonType (8d36a78440), PtrTo support from gc/ld (44fd7573aa), type-kind constants moved to a separate file (ede6718cd7), an unused gc field (65e61d5770, Go 1.1 “unused for now”). In Go 1.1 reflect.ArrayOf/ChanOf/MapOf/SliceOf required finding existing types in the binary, so the linker emitted a list of such types and the interface{} header was dropped from the runtime type representation, a net -156 kB in godoc (1120982590).
Between Go 1.3 and 1.5 the GC information evolved: the 2-bit “sparse” compiler format (cd17a717f9, Go 1.4, -25% memory, -3–7% binary size), reflect: generate unrolled GC bitmask (3cf9acccae), kindNoPointers/KindDirectIface flags (1806a5732b), and 1-bit pointer bitmaps in type descriptors and GC programs (6d8a147bef, Go 1.5: “1 bit per word, down from 2”). ptrdata size was added (98a9d36837), kindNoPointers was removed in Go 1.13 (db16de9203).
Go 1.7 is the descriptor-shrinking release, driven by David Crawshaw (issue 6853) and measured as binary-size cuts: the string header embedded in rtype (a858931200, cmd/go -1%), uncommonType.name removed (0231f5420f, -0.5%), ptrToThis removed (30f93f0994, -0.2%) then brought back for reflect.New speed (e369490fb7, 36.3→29.5 ns), the *uncommonType pointer replaced by a flag with the contents following the rtype (a24b3ed753, godoc -180 KB), slices removed from funcType (8df733bd22, godoc -2.4%; limits of 65535 inputs, 32767 outputs, FuncOf 128), the second method type computed at run time with reflect.FuncOf (f2772a4935, jujud -5.6 MB), and a new static name encoding (24ce64d1a9): first byte bit field (exported, tag follows, pkgPath follows), two-byte big-endian length, string data, optional tag, optional import-path pointer; cmd/go -164 KB. Method tables, names, pkgPath, rtype string and uncommonType were converted to offsets: typeOff (7d469179e6), nameOff in method tables (95df0c6ab9), pkgPath names (f120936dff), rtype string (1492e7db05), uncommonType (c165988360), with go.string.* free of pointers (4d920410d2) and unexported method type info removed (9f38796270, 44d3f89e99). tflag-based Name() fast paths (797dc58457, ed9362f769, e75c899a10) recovered JSON-decode regressions; StructOf clears tflag (4b64c53c03), and many-field structs needed an extra 8 bytes for every type with methods (af0fc83985).
Multi-module resolution (buildmode=shared/plugin): typeOff is relative to firstmoduledata.types; for other modules the runtime keeps a typemap that maps duplicates to the canonical rtype so each type has one *rtype, and reflect-created types use the offset as a global ID in a runtime reflect offset map (7d469179e6: linux/amd64 PIE jujud -3.5 MB, 4.2%). Go 1.8 made typelinks init go forwards (bee4206764), made typemaps visible to the GC (f4c7a12c2c), resolved offsets using the source module (34c480afa8) and wrote type symbols referenced in ptabs (8eb9fdaa01, 5923df1af9). Go 1.14 fixed textOff with multiple text sections (0ae9389609), Go 1.16 used a sentinel offset -1 for unreachable methods (0ab72ed020), Go 1.17 made names longer than 65535 possible with a varint length (287025925f).
Timeline.
- Go 1.0 — Done:
- 2008: reflection from type strings; sys.Reflect/Unreflect (
d6a7cefd1e,a45f947c34,842e1a9aa7). - 2009: struct-based reflect data laid out by the compiler (
4793400bd1,a7b4e9f03e,58c4142e74), maps and channels reflected (764b6ec1aa,5ddaf9a098), methods (12ebbe7463). - 2010–2011:
Kindreplaces per-type structs (45bdf0367e), gc emits correct reflect types (96db724cfc),ptrtoword (8d36a78440), uintptr treated as pointer-bearing (b0c674b65d), runtimeTypedeleted (121a16fa95,6a75ece01c), type-specific algorithms (b9ccd077dc). Left open: type data still lacked GC info.
- 2008: reflection from type strings; sys.Reflect/Unreflect (
- Go 1.1 — Done:
gcfield (65e61d5770), array/chan/map/slice type list and header removal (1120982590), types of allocated objects stored (4a191c2c1b),Itabmoved toruntime.h(5c1422afab), GC info for reflect-made types (3660b532ac). Left open: GC info onArrayOf(issue 4375). - Go 1.3 — Done:
zerofield in rtype (aad4609c08), precise reflect Value support (cbc565a801). - Go 1.4 — Done: sparse 2-bit GC type info (
cd17a717f9), direct-iface flag in kind (1806a5732b), unrolled GC bitmask in reflect (3cf9acccae),Type.Comparable(a325f4f2b3), type.go duplicate removed (81674f3135). - Go 1.5 — Done: 1-bit bitmaps (
6d8a147bef), pointer size (98a9d36837),PtrTofinds existing pointer types by string before synthesizing (2f0828ef7c, needed for dynamic linking), method-name equality instead of identity (e1366f94ee),kindNoPointerson function layouts (d11f411181). - Go 1.6 — Done:
zerofield removed (af78482d6b),_Kindconstants removed (2961cab965), methods sorted withsort.Interface(bca70a66a7). - Go 1.7 — Done: the shrinking series and name encoding above (
a858931200,0231f5420f,30f93f0994,a24b3ed753,8df733bd22,f2772a4935,24ce64d1a9,7d469179e6,95df0c6ab9,f120936dff,1492e7db05,c165988360,4d920410d2,e369490fb7,797dc58457,af0fc83985,43a274fd67pkgPath on all struct types,c9638810df). Left open: “further shrink run time type information in Go 1.8” (e369490fb7). - Go 1.8 — Done: typemap/typelinks work (
bee4206764,f4c7a12c2c,34c480afa8), qualification of unexported fields of unnamed types (14efaa0dc3), field pkgPath (011cb64231,8d0c105407), Noalg types no longer break user types (3797446150), main-symbol type table (5923df1af9). - Go 1.9 — Done: dxxx functions converted to
obj.LSym(4d600b8e5f,bea8ffdbb1,b065c95551,08db34927d,2f4ce69ddd,0d1e903c8a,1dc3b60bd6,3cf72322dc),dumptypestructssplit (976a5ce1a8,92607fdd30),signatlistmap with deterministic sorting (91433eb577reverted by re-roll6d9c77f5dd,6f2ee0f3df,e577a55b78,25fc842f17),ptrdatadocumented (2ef340f271), plugin recursive types (e49627d355), exported flag in namedata (ee392ac10c). - Go 1.10 — Done: struct field reflect info fixes (
d349fa25df), Noalg types out of typelinks (a4c009f5ae),dtypesymreturns*obj.LSym(d36cc9baf3). - Go 1.11 — Done: method sets sorted earlier and by package height (
91bbe5388d,49ed4cbe85,86a338960d), unusedSig.offsetremoved (fd9d2898bd), deterministic anonymous-interface output (876c6d1f27). - Go 1.13 — Done:
kindNoPointersremoved (db16de9203). - Go 1.14 — Done:
fieldalignnaming (9c0e56bf9d), textOff for multiple sections (0ae9389609). - Go 1.16 — Done: GC-map and itab symbols content-addressable (
17344d55d6,cecb7a1cf3), unreachable-method sentinel (0ab72ed020), exported/non-exported interface method split (8f26b57f9a, len = exported, cap = all), pointers togo:notinheaptypes stored indirectly (009d714098). - Go 1.17 — Done: varint name lengths (
287025925f), GC prog symbols content-addressable (e99dfb0e5c), no reflect data for method types (1d9a1f67d5),typesByStringuses all methods (2169deb352),unsafe.Pointerpackage path fix (40254ec0db),reflectdata.WriteTypeunexported andLinksymOffsetExprforTypePtr/ItabAddr(03a875137f,4c835f9169), bucket type tweak (15085f8974),internal/abicreated (baa6c75dce).
Interface values, itabs and type assertions
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (compiler-emitted itabs, runtime itab hash table) → continues as Interface conversion: itabs, type switch and assertion caches (1.19+)
Paths not taken: GOEXPERIMENT=largeiface (never landed)
- proposal #14874 proposal: cmd/compile: compiler-emitted itab symbols referenced by 2 of this thread's commits
- motivating issue #20505 runtime: need a better itab table referenced by 2 of this thread's commits
Design. src/runtime/iface.c (2008) implemented interface conversion, with new interface error messages (6f07ec721a), interface equality (e2e8aa0ee1) and zero-interface handling (900e076d21); interfaces holding interfaces were disallowed in 2009 (4b8c13dc20). An Itab caches the mapping from (interface, concrete type) to method pointers; in Go 1.2 itabs moved from mallocgc to persistentalloc (b36f2db12a). The Itab struct moved to runtime.h (5c1422afab).
Go 1.4 decided the “direct interface” policy in the compiler: isdirectiface in cmd/gc/subr.c is the only place deciding which types are stored in the data word, and the policy propagates through reflect metadata as KindDirectIface (1806a5732b). It was done so the representation could change for the concurrent GC (no ambiguity whether the data word is pointer or scalar); the follow-up made interfaces always hold pointers (33e910296e in reflect, [dev.cc]). The iface code was converted to Go (483cb61921, b53b47f5ac). Go 1.5 made _, ok := i.(T) inlinable (25e793d7ea, assertion benchmarks -80%) and fixed uninitialized memory in assertE2I2 (c4092ac398) and interface comparison (05a3b1fce5). Go 1.6 made iface/eface handling type-safe by taking iface/eface parameters (84afa1be76) and instrumented convT2E/I for race (9172a1b573).
Go 1.7 shifted work to the compiler: itab symbols are emitted by the compiler and convT2I is a two-word copy for pointer-shaped types (7427f2c4bd), with the runtime inserting compiler itabs into its hash table at init (7043d2bb5e), and convT2{I,E} no longer handling direct interfaces (998c8e034c); getcallerpc fixes (db44223fde, 260b7daf0a); wrong-panic fix (09834d1c08). Go 1.8 had to cope with shared-library duplicates: go.itab.* symbols were first module-local (8b07ec20f7, issue 17594: the same itab added twice created a list cycle), then made global again with additab called once per itab (1ec64e9b63, issue 18252), with a mark-local-earlier compiler tweak (4e584c5203) and statically initialized interface values (dfc56a4cd3). Go 1.10 set itab function pointers at compile time (89d74f5416, shorter startup) and replaced the linked list with a growable open-addressed hash table with quadratic probing and 75% max load; the read path takes two atomic loads (3d1699ea78, BenchmarkItabAll 228 ms to 4.3 ms with fcf445dce2), removing bad and link fields (98d0634b7a, 04d6f982ae), initializing itab.hash always (fcf445dce2), and not publishing a table before growth finishes (d30f99647a). Go 1.13 sped up ifaceeq for direct interfaces (694cd005c3, EfaceCmpDiff -29%) and set itab.fun[0] only on success (8d86ef2216). Go 1.15 avoided allocation when converting small integers to interfaces via staticuint64s (9828c43288; compiler -S allocs -14%). Go 1.16 made itab symbols content-addressable (cecb7a1cf3, requiring global uniqueness since pointers are compared).
Timeline.
- Go 1.0 — Done: 2008–2009: iface runtime (
6f07ec721a,e2e8aa0ee1,900e076d21,4b8c13dc20); 2010–2012: type switch eliminates package-global-name assumption (1912632019),runtime·Typedeleted. - Go 1.1 — Done:
Itabin header (5c1422afab), types of allocated objects (4a191c2c1b). - Go 1.2 — Done: itabs from
persistentalloc(b36f2db12a),SetFinalizeraccepts any compatible f (5822e7848a), wrappers for blank-interface methods skipped (f316a7ea87). - Go 1.3 — Done: interface data correctly typed (
e9977dad45), scan of pointer-in-interface same as pointer (68aaf2ccda), wrapper for(interface{...}).F(f303921251). - Go 1.4 — Done: direct-iface decision in compiler (
1806a5732b), iface in Go (483cb61921,b53b47f5ac), reflect interface allocation type (7b9c5ec24b: GC saw empty-interface layout for non-empty interface),x.(T)for 0-byte T (d646040fd1). - Go 1.5 — Done: inlined
_, ok := i.(T)(25e793d7ea), assertion fixes (c4092ac398), typed memmove for interface contents (54bb4dc390). - Go 1.6 — Done: type-safe iface/eface (
84afa1be76),convT2E/Irace read (9172a1b573). - Go 1.7 — Done: compiler itabs (
7427f2c4bd,7043d2bb5e,998c8e034c),getcallerpcargument fixes. Left open: shared-library duplicates. - Go 1.8 — Done: duplicate itab handling (
8b07ec20f7,1ec64e9b63), static iface init (dfc56a4cd3). - Go 1.10 — Done: compile-time function pointers (
89d74f5416), hash table (3d1699ea78,98d0634b7a,04d6f982ae,fcf445dce2,d30f99647a). - Go 1.11 — Done: better assertion panic messages for identical-looking types (
75c1aed345,cda1947fd1),printanyno longer checks String/Error (804e3e565e). - Go 1.13 — Done:
ifaceeqdirect fast path (694cd005c3),fun[0]only on success (8d86ef2216). - Go 1.14 — Done:
efaceOfused (64e598f783),itab.hashof dynamic entries documented unused (61089984a8). - Go 1.15 — Done: small-int conversions without allocation (
9828c43288), interface hashing (afd691c579), more printable panics: values of named basic kinds are printed (972df38445). - Go 1.16 — Done: content-addressable itab symbols (
cecb7a1cf3), plugin-exported types marked used in interface (86954d5246). - Go 1.17 — Done: register-ABI-aware paths (
849dba07a5). Left open: type switches and assertions had no per-site cache yet (appears in 1.19+).
reflect redesign: from string parsing to Type/Value structs
Span: Go 1.0 → Go 1.4 · Status at go1.17: still the design at go1.17 for the Type/Value API (opaque Value {typ, ptr, flag}); implementation tuned in 1.18+ → continues as reflect implementation: performance, allocation and escape behaviour (1.18+)
Paths not taken: A two-word reflect.Value (never landed)
Design. reflect in 2008 parsed type strings, with separate value types per kind (IntValue, PtrValue, …). The July 2009 rewrite replaced string parsing with compiler-laid-out structs (4793400bd1, a7b4e9f03e, 58c4142e74) and added chan/map/method reflection (764b6ec1aa, 5ddaf9a098, 12ebbe7463). 2010 added Kind (45bdf0367e) and Type.Bits. On 2011-03-03 PtrTo/Value.Addr made it possible to take addresses to call pointer-receiver methods (e46acb091f, old Addr renamed UnsafeAddr). In April 2011 reflect moved to Type as an interface implemented for all kinds and Value as a struct (fb175cf77e, many methods renamed: Elem to Index/MapIndex, Get to Int/Float…, v != nil to v.IsValid()); the motivation was an allocation-free Value, with the first CL a wrapper over the old API. Subsequent steps: inline methods (7b6ee1a5d4), Set semantic change so that assignable values always refer to other memory (“cannot Set result of NewValue anymore”, gob test mallocs 6/6 → 3/5, 40fccbce6b), assignability matching Go plus Implements, AssignableTo, CallSlice (e1ee3b5db6), and opaque Value struct in Nov 2011 (a479a45548, JSON benchmarks about 1.8x faster). Struct tags became key:“value” convention with StructTag.Get (25733a94fd). Other 1.0 API: MethodByName, Value.Bytes/SetBytes, Value.NumMethod, Zero, PtrTo, FieldByName with embedded structs and FieldByIndex.
Value layout continued to change under GC pressure. Go 1.3 rewrote Value to separate pointer from non-pointer information, making it 4 words, “needed for precise gc and copying stacks” (cbc565a801); reflect select/chan ops stopped using interface words (873aaa59b7, one allocation per receive case). Go 1.4 shortened it back to 3 words, scalar no longer needed because interfaces always hold pointers (a1616d4a32).
Timeline.
- Go 1.0 — Done:
- 2008: string-parsing reflect,
Value.Interface, tags (554d0aa589,12a3435869,a45f947c34), type cache (842e1a9aa7),DeepEqualin 2009 (c4ad4f9fcf). - 2009: struct-based rewrite (
4793400bd1,a7b4e9f03e,58c4142e74), map/chan/method reflection (764b6ec1aa,5ddaf9a098,12ebbe7463),FieldByNamethrough embedding (a288095813),DeepEqualfor maps (1880b90aff). - 2010:
Kind(45bdf0367e),Type.Bits(fc090a3a54),Append/AppendSlice(8b64cd9c5e),FieldByNameFunc(57e764171c). - 2011:
PtrTo/Addr(e46acb091f), new Type/Value (fb175cf77e,7b6ee1a5d4,40fccbce6b,e1ee3b5db6), struct tag convention (25733a94fd),MethodByName(22484e2262), opaque Value (a479a45548),Interfaceon unexported values disallowed (304cf4dc9b), empty slice/map not DeepEqual to nil (4e65478cbd). - 2012:
DeepEqualon funcs (3a1c226a38),Value.Interfacereturns immutable data (a72b87efa9), MakeSlice argument checks (11cc5a26d5).
- 2008: string-parsing reflect,
- Go 1.1 — Done: user-visible
Select(370ae05545),Convert/ConvertibleTo(46f379cc2c),MakeFunc(ba4625c66f),ArrayOf/ChanOf/MapOf/SliceOf(1120982590), method values (3be703665e), fasterFieldByNamevia BFS (5e3224ce79,FieldByName38.2 ms → 42 µs;aa38801788multiplicity fix),Index/Sliceon strings (772decbc80),Zeronot addressable (6044dbdf1b), no run-time code generation (b1b67a36ac). - Go 1.2 — Done: user-visible
Value.Slice3,SetCap(4d8aefde47);Interfaceof non-addressable no longer allocates (94179d61ab,InterfaceBig90→14 ns),Convertindir-bit fix (0e73497a4b),DeepEqualvisit key (f7b5a01999,780f5b714d), call arguments exposed to GC (0ab8f2d287). - Go 1.3 — Done: Value rewrite for precise GC (
cbc565a801), precise GC info forCallframes (2af7a26f1e), channel/select without iwords (873aaa59b7),Zeroallocation fix (e7d899cba5), embedded unexported fields read-only (a8a7f18aea),MakeFuncvariadic-argument fix (772d22885b), reflectCallof interface method uses the correct type descriptor (fcf8a77525), nil-map delete no longer panics (cb6cb42ede). - Go 1.4 — Done: 3-word Value (
a1616d4a32),Type.Comparable(a325f4f2b3), fastFieldByIndex(94950afdf8), struct size includes trailing padding (4e1d196543), memmove shared with runtime (b78d7b75c7), a few microoptimizations (0d81b72e1b).
reflect dynamic types: ArrayOf, ChanOf, MapOf, SliceOf, FuncOf, StructOf
Span: Go 1.1 → Go 1.14 · Status at go1.17: still the design at go1.17 (synthesized rtype structures with GC info built in reflect) → continues as reflect implementation: performance, allocation and escape behaviour (1.18+)
Design. To create []T, chan T, map[K]V at run time reflect first looks for an existing instance: the linker emits a list of such types (1120982590, “typelinks”), and only if absent synthesizes a new descriptor. Go 1.1’s ArrayOf was unexported pending garbage-collection support and a correct algorithm (equality and copy), tracked as issue 4375; GC information was added for ChanOf, MapOf, PtrTo, SliceOf (3660b532ac). PtrTo of an unnamed type with embedded fields was fixed in Go 1.5 (7feb424928) and the pointer lookup consults existing type data by string before synthesizing, required for dynamic linking (2f0828ef7c). Go 1.5 added FuncOf (e1c1fa2919, d36cc02795 for PtrTo(FuncOf)) and exported ArrayOf (918fdae348, e00e65638a) and ChanOf direction (48d63035ce).
Go 1.7’s StructOf (633ab7426a) creates new struct types from StructFields in the form structTypeWithMethods to accommodate the GC; embedded interfaces with unexported methods and non-ASCII names were initially unsupported (+0.04% bin/go, reflect +6.2%). The restriction on UTF-8 names was lifted (9d36cac99c), methods that cannot work panic (87d5f6b9f6, Go 1.9), invalid field names rejected (538b3a5f37), embedded method cases fixed (83092a40ac, 0e4a0b93d2), GC programs for StructOf fixed in Go 1.13 (05b3db24c1, 9dc3b8b722, b7b7b4d6d4), unexported fields allowed in Go 1.14 (2d1c033259) and PkgPath recorded (ba66797392). Other: ArrayOf with zero-size element (9bced47706), SliceOf/ArrayOf string fixes (bddfc337f9, a4864094f0), ChanOf string parenthesization (407bf0ca67).
Timeline.
- Go 1.1 — Done: user-visible
ArrayOf(unexported),ChanOf,MapOf,SliceOf(1120982590), GC info for synthesized types (3660b532ac). Left open: GC and algs forArrayOf(issue 4375). - Go 1.3 — Done: correct map type generation (
5bc1cef869). - Go 1.5 — Done: user-visible
FuncOf(e1c1fa2919),ArrayOf(918fdae348), pointer lookup (2f0828ef7c),ChanOfdir (48d63035ce), call-frame GC bitmap uses two bits per pointer (7c1e33033d). - Go 1.7 — Done: user-visible
StructOf(633ab7426a), unnamed directional channels (69285a8b46),SelectDir(368507bb6f),StructTag.Lookup(5c8674a497), unexported-methods hidden (be1b930653). Left open: embedded fields with methods unsupported (905ced0e6a). - Go 1.9 — Done:
MakeMapWithSize(6c5a819a5e),StructOfmethod checks (87d5f6b9f6,2ddc3e940e). - Go 1.13 — Done: StructOf GC program fixes (
05b3db24c1,9dc3b8b722,17f888c5a8). - Go 1.14 — Done: unexported fields, PkgPath (
2d1c033259,ba66797392). - Go 1.17 — Done: user-visible
ArrayOfpanics on negative length (d7cc2f1d7c),ArrayOf/StructOfregressions7473a6a0eb(stack overflow inhaveIdenticalUnderlyingType).
reflect.Call, MakeFunc, method values and argument-frame GC precision
Span: Go 1.1 → Go 1.17 · Status at go1.17: replaced by the register-ABI implementation in 1.17 (see register ABI thread) → continues as Register ABI completion for reflect, memmove/memclr and bytealg across architectures (1.18+)
Design. reflect.MakeFunc (ba4625c66f, 1.1) first generated code at run time through makeFuncStub, an assembly trampoline per architecture; “Step 3 of go11func” (b1b67a36ac) stopped the run-time code generation. Method values (3be703665e) use methodValueCall with a methodValue context struct. Frames: reflect.call assembled the arguments into a callXX frame; in Go 1.3 callXX routines no longer call the stubs directly but inline their behavior (85138da832), the argument frame gets precise GC info (2af7a26f1e) and in Go 1.4 call frames are cached with sync.Pool (67f8a81316, 634→338 ns, 1→0 allocs). The runtime needs to find the argument maps of these stubs during traceback (f0d44dbeaf, d5bd797ee5). Go 1.12 fixed a subtle bug (issue 27695): results in the frame must not be scanned at the start of a call (uninitialized) but must be scanned at the end; a boolean slot in the makeFuncStub/methodValueCall frame tells the runtime, and the bitmap always covers args plus results (ef50373983). Related: write-barrier fixes for method funcs (e35a41261b), callReflect result copy without write barrier (7e987b7b33), and Go 1.15 zeroing stack slots before barrier writes (3dec253783).
Timeline.
- Go 1.1 — Done: user-visible
MakeFunc(ba4625c66f), method values (3be703665e), run-time code generation removed (b1b67a36ac). - Go 1.3 — Done:
callXXdo not call stubs (85138da832), precise frame GC (2af7a26f1e), variadic MakeFunc fix (772d22885b), align call args on amd64p32 (ea7d801130), interface-method call type descriptor (fcf8a77525). - Go 1.4 — Done:
sync.Poolframe cache (67f8a813161.5), stub arg maps in traceback (f0d44dbeaf),assignTo/cvtT2Iallocate correct type (7b9c5ec24b). - Go 1.5 — Done: correct
funcLayoutbitmap (7c1e33033d),kindNoPointersfor pointer-free layouts (d11f411181),mapassignnoescape (84e2567537). - Go 1.8 — Done:
getArgInfofor deferred reflection calls (d5bd797ee5),makeFuncImplkept live across stub (22689c4450). - Go 1.9 — Done: out-of-bounds pointer fix for no-result method calls (
627798db4e). - Go 1.10 — Done: interface→interface conversion in
Call(eafa29bdce), zero-sized return value pointer (0a48185b43). - Go 1.12 — Done: results scanning fix (
ef50373983), method-func write barriers (e35a41261b), s390x method calls (0b994a9037), no write barrier copying results (7e987b7b33). - Go 1.13 — Done:
MakeFuncassignment conversion of returned values and better panic forValue{}(fe83731651,04845fe78a),callMethodaligns first argument (5f45a3337e). - Go 1.15 — Done: zeroing before barrier writes (
3dec253783),Selectallocations on stack for up to 4 cases (17d5cef257, Select/1 -18%). - Go 1.16 — Done: pending register ABI:
internal/abiconstants zero under regabi (8cfbf34dd91.17). See register ABI thread. - Go 1.17 — Done: register-ABI reflect.Call (see next thread).
Register ABI support in reflect and runtime assembly (Go 1.17)
Span: Go 1.17 → Go 1.17 · Status at go1.17: reflect assumes the register ABI only under GOEXPERIMENT=regabiargs and uses global register counts; not yet default → continues as Register ABI completion for reflect, memmove/memclr and bytealg across architectures (1.18+)
- tracking issue #40724 cmd/compile: switch to a register-based calling convention for Go functions referenced by 8 of this thread's commits
- design doc Proposal: Register-based Go calling convention Austin Clements, 2020-08-10 · linked from #40724
Design. internal/abi was created in the [dev.regabi] branch to hold ABI constants shared by runtime and reflect (baa6c75dce). reflect implements the register-assignment algorithm itself (28c5fed557 adds makeFuncStub/methodValueCall support): stack-assigned arguments follow the spec and compiler; zero-sized types need their alignment counted in stack offsets (a2f7067233) and the two short-circuits for zero-sized types were removed because an array of non-zero length of zero-sized elements must be stack-assigned (34b87b4a1a); spill space calculation fixed (cda8ee095e), counts undone on register assignment failure (51cd074c59), pointer register results kept alive in callMethod (ef36e4fd0e), stack-to-register translation in callMethod (9d46ee5ac4), ctxt preserved across moveMakeFuncArgPtrs (4fb74e0555), methodValueCall frame size on mips64 (41cf18eda7), and callReflect/callMethod are called as ABI0 (d446cb7cff).
reflect uses global register counts rather than the internal/abi constants so those constants can be non-zero (enabling the runtime’s spiller/unspiller) while reflect still works in ABI0, “turning two things we need to flip into one” (c5a1c2276e, 8cfbf34dd9). 27d306281c makes finalizers, reflect calls and Windows syscall callbacks assume the register ABI when GOEXPERIMENT=regabiargs is set. For the runtime and bytealg assembly, 849dba07a5 ports performance-critical functions (memmove, memclr, memequal, cmpstring-like) to read register arguments directly so builtin calls from compiled ABIInternal code avoid ABI wrappers (wrappers cost on many benchmarks, e.g. BiogoKrishna -4.6%); 8f4c5068e0 does the same for internal/bytealg; compile-time memequal_varlen is referenced as ABIInternal (411860251e) and method wrapper tail calls are disabled under RegabiArgs (45e87cd3ec).
Timeline.
- Go 1.17 — Done:
internal/abi(baa6c75dce), reflect register algorithm (28c5fed557,a2f7067233,34b87b4a1a,cda8ee095e,c5a1c2276e,51cd074c59,ef36e4fd0e,9d46ee5ac4,4fb74e0555,41cf18eda7,d446cb7cff), regabiargs assumption (27d306281c,8cfbf34dd9), wrapper avoidance (849dba07a5,8f4c5068e0,411860251e,45e87cd3ec). Left open: register ABI stays an experiment; reflect register count variables must be flipped when it becomes default (message ofc5a1c2276e).
reflect performance, caches and API additions
Span: Go 1.4 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as reflect implementation: performance, allocation and escape behaviour (1.18+)
Design. After the Go 1.4 Value shrink, the focus was cost on hot paths (JSON decoding, protobuf). Caches moved from RWMutex maps to sync.Map in Go 1.9, giving large multicore speedups (Call-48 133→12 ns, CallArgCopy +13x, 33b92cd6ce). Name() was optimized using tflag (797dc58457, ed9362f769, e75c899a10), NumMethod()==0 avoids a lock (3c6ed76da2, CodeDecoder -8%), directlyAssignable checks tflagNamed rather than calling Name (8057c0887f, 32.7→6.6 ns), flag.mustBe* methods were made inlinable (788e038e5d), reflect.Zero uses a shared zero buffer under 1 KB and Set uses memclr when the source is known zero (8925290cf7), IsVariadic cached in Call (6cadfe2fee, ~-9%), Swapper added in Go 1.8 (ca04091f5b), Value.MapRange/MapIter in Go 1.12 (ede5958385), IsZero in Go 1.13 (c40bffd905), StructField.IsExported, Method.IsExported (b83d073e9e), VisibleFields (009bfeae86), slice→array-pointer Convert that makes ConvertibleTo true although Convert may panic (760d3b2a16), CanConvert (48c88f1b1b), and struct tags with multiple keys (5d12434eee), all Go 1.17.
Timeline.
- Go 1.4 — Done: three-word Value, FieldByIndex fast path, microoptimizations (
a1616d4a32,94950afdf8,0d81b72e1b). - Go 1.5 — Done:
mapassignnoescape (84e2567537), struct tag key validation (6ea3adc3ba),DeepEqualunsafe pointers. - Go 1.7 — Done: tflag/Name speedups (
797dc58457,ed9362f769,e75c899a10,3c6ed76da2),ptrToThisback (e369490fb7), unnamed types have no PkgPath (1f4e68d92b). - Go 1.8 — Done:
Swapper(ca04091f5b), zeroing avoided (adb1e67f02), struct tags ignored in struct conversion (ddb77100a6), DeepEqual cycles (19adf8aeaa). - Go 1.9 — Done:
sync.Mapcaches (33b92cd6ce),MakeMapWithSize(6c5a819a5e),bucketOffix (b5e5194306), clearptrToThis(066ac428cd). - Go 1.10 — Done:
Copyfrom string (245e386e4c), safety audit of unsafe additions (8bb51a73e9), method indexing for non-ASCII (6f1724ff41,77b4beba2f). - Go 1.11 — Done: avoiding
commonwhen type is*rtype(1dbe4c50f2), terminology defined/embedded (47be3d49c7), DeepEqual map reuse (c4f9fa1f8e), exported methods sorted first (86a338960d). - Go 1.12 — Done:
MapRange(ede5958385), UnsafePointerIsNilfix (3b66c00857). - Go 1.13 — Done:
IsZero(c40bffd905), inlinablemustBe*(788e038e5d), method sets documented sorted (f33b67b870). - Go 1.14 — Done:
directlyAssignablespeedup (8057c0887f), type hints in error messages (09824ccfe5), DeepEqual cycle panic fix (4dc11ae26b). - Go 1.15 — Done: float32 conversion keeps signaling NaNs (
7485050c0f,7ffbea9fd8), unexported embedded method invocation disallowed (0eb694e9c2),Value.AddrRO flags (4c003f6b78),unsafeheaderconsolidation (da382a3978), DeepEqual flagIndir (d872bbcfec). - Go 1.16 — Done:
Zerobuffer (8925290cf7), DeepEqual depth removed (77a11c05d6), multiple struct tag keys (5d12434eee), Convert int-to-string regression fix (f0ff6d4a67). - Go 1.17 — Done: user-visible
VisibleFields(009bfeae86),IsExported(b83d073e9e), slice→array pointer conversion (760d3b2a16),CanConvert(48c88f1b1b),IsVariadiccache (6cadfe2fee),Newof ago:notinheaptype panics (554d2c4f06); shared conversion panic (0fd20ed5b6), documentation of convertibility (cf4e3e3d3b). Left open:Valuestill escapes in many paths (addressed in 1.18+ per the later threads).
memmove, memclr, duff routines and pointer-write atomicity
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as memmove / memclr / memequal / internal/bytealg assembly optimisation (1.18+)
Design. Go 1.1 removed a memset/memclr shim (3a50bc1a24) and fixed amd64 memmove overflow (1b8f51c917). Go 1.2’s x86 memmove (6021449236): REP instructions have a high startup cost, so small sizes are handled with straight-line code; REP MOVSx is used for large sizes with a ~1 KB cutover; up to 128/256 bytes uses SSE loads of all data before any stores, which makes copy direction irrelevant. Go 1.3 replaced REP STOSQ in memclr with explicit SSE writes (da7cf0ba5d, Memclr5 22→5 ns). Go 1.4 widened MOV-SSE copies to 257–2048 bytes (a712e20a1d) and removed a pointless JMP-to-RET (bc9e160443); memclr memclrBytes to Go (68f91aea57). Go 1.5: auto-generated duff routines (ad3600945a, mkduff.go), faster duffzero mixing STOSQ and MOVQ/ADDQ blocks (7e0c11c32f), MOVUPS zeroing (2421c6e3df), duffcopy with movups (5cf281a9b7). Go 1.6 / Go 1.14 added the rule that memmove/memclr must write pointer-sized units atomically because the concurrent GC may observe partial pointers (issue 12552, 4b7d5f0b94, ppc64 and arm64 word copy in 2c911143fd, 168a51b3a1, amd64p32 4-byte writes 9f6df6c940, arm64 ffbc02761a in 1.14, documented in 9b5bd30716, memclrNoHeapPointers atomicity documented in 1.16 by 68058edc39). Go 1.7: MOVSB for unaligned moves (6a33f7765f, UnalignedDst4096 -41%) but not when CPUID lacks fast REP MOVSB (4b209dbf0b); ppc64 memmove speedups (c4807d4cc7). Go 1.8: ppc64x memclr (3107c91e2d), 4-byte move special case to avoid store-forwarding stall (a96e117a58, Issue18740 -36%). Go 1.10–1.12: arm64 memmove/memclr SIMD, ppc64x VSX with dcbz (d259815ccb, 213a75171d, aa9bcea390); jump tables rejected (f864d89ef7). Go 1.15: ppc64 short moves (cbd421f75b), MIPS64x memclr with MSA (918d4d46cd), duffcopy dummy for ppc64 (7939c43748). Slice-specific uses are covered in the slice thread.
Timeline.
- Go 1.1 — Done: shim removed, overflow fix (
3a50bc1a24,1b8f51c917), ARM memmove register avoidance (e34079bb59). - Go 1.2 — Done: fast x86 memmove (
6021449236), ARM memmove parameter corruption fix (f8fd77baa9). - Go 1.3 — Done: faster x86 memclr (
da7cf0ba5d), Plan 9 uses unoptimized versions (8303a13bb8). - Go 1.4 — Done: amd64 memmove (
a712e20a1d), no pointless jmp (bc9e160443),memclrBytesin Go (68f91aea57), reflect uses runtime memmove (b78d7b75c7). - Go 1.5 — Done: duff routines generated and tuned (
ad3600945a,7e0c11c32f), memmove in append (497970f421). - Go 1.6 — Done: duffzero/duffcopy amd64 (
2421c6e3df,5cf281a9b7), pointer-atomic memmove/memclr (4b7d5f0b94,2c911143fd,168a51b3a1,9f6df6c940), ppc64 NOFRAME (45c06b27a4). - Go 1.7 — Done: MOVSB (
6a33f7765f,4b209dbf0b), ppc64 (c4807d4cc7). - Go 1.8 — Done:
3107c91e2d,a96e117a58. - Go 1.10 — Done: arm64 SIMD IndexByte/memclr (
d259815ccb). - Go 1.11 — Done: amd64 3/4-byte moves simplified (
910d232a28), arm64 memmove (213a75171d), jump-table TODOs removed after measurement (f864d89ef7). - Go 1.12 — Done: ppc64x VSX memclr/memmove (
aa9bcea390). - Go 1.14 — Done: arm64 pointer atomicity (
ffbc02761a), requirements documented and tested (9b5bd30716), wasm zero/move (d0f10a6e68). - Go 1.15 — Done: ppc64 short moves (
cbd421f75b), MIPS64x memclr (918d4d46cd),7939c43748. - Go 1.16 — Done: memclrNoHeapPointers atomicity documented (
68058edc39). - Go 1.17 — Done: ABIInternal ports (
849dba07a5).
internal/bytealg and string/byte comparison primitives
Span: Go 1.2 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as memmove / memclr / memequal / internal/bytealg assembly optimisation (1.18+)
Design. Faster equality and compare started in the runtime: b3946dc119 (Go 1.2: fast Compare for byte arrays and strings), 3d5daa2319 (Go 1.1, amd64/386 string and byte equality), 77f3e189d2 (identical-pointer fast path). Go 1.4 compared only up to min(len) (5ea131f442); Go 1.5 added runtime.cmpstring and bytes.Compare in assembly on arm (e2543ef62c) and arm64 (ee349b5d77), then ppc64 in Go 1.6 (32d3b96e8b) and a short-circuit when the slices are identical (805e56ef47). In Go 1.11 the assembly was consolidated in a new internal/bytealg package: IndexByte (403ab0f221), equality (1dfa380e3d), Count (45964e4f9c), Compare (f6332bb84a), short-string Index (ee58eccc56), wasm (b382fe28a7), and it dropped the dependency on runtime·support_avx2 (f8b28e28f8), with explicit argmaps (be9c994609), linkname prototypes (ef7ce57ac2) and later inlineable trampolines instead of linkname (ad4a58e315, Go 1.12). Per-architecture work: arm64 Compare (bfa8b6f8ff, d31ee64186, +750% for big), Index (d4e936cfd6, f8ef6ed24a), Equal (de28555c0b); arm word-wise Compare/Equal and shared IndexByte, Count (029a5af6a1, a734601bdf, 14a58d65e3, 3496ff1d19, 3cb1e9d98a); ppc64x Count/IndexByte/memequal (a0fad982b1, 23f7554194, 6994731ec2); mips64x Compare (78baea836d, a1b1ba7daf after a revert for a little-endian bug, 4f597abe77); s390x SIMD Count (b6245cef3c); riscv64 (0c703b37df, 19f6422e00); wasm (d0f10a6e68, 4a7cd9d9df); Rabin-Karp moved to bytealg (18a6fd44bb) and faster IndexAny (3af92acb9f); Index asm for ppc64x (8009a81f7a, 54af9fd9e6 for power9) and CBZ (99d6e3eec2); IndexString misuse fix (11cdbab9d4). In Go 1.17 the functions were ported to ABIInternal (8f4c5068e0).
Timeline.
- Go 1.1 — Done: amd64/386 faster string and byte equality (
3d5daa2319,77f3e189d2). - Go 1.2 — Done:
Comparefor bytes/strings (b3946dc119). - Go 1.4 — Done: compare only to min length (
5ea131f442). - Go 1.5 — Done: arm and arm64
cmpstring(e2543ef62c,ee349b5d77). - Go 1.6 — Done: ppc64 and identical-slice shortcut (
32d3b96e8b,805e56ef47). - Go 1.10 — Done: arm64 IndexByte/memclr SIMD (
d259815ccb). - Go 1.11 — Done:
internal/bytealgcreated and populated (403ab0f221,1dfa380e3d,45964e4f9c,f6332bb84a,ee58eccc56,b382fe28a7,63bcabed49,f8b28e28f8), arm64 Compare/Index (bfa8b6f8ff,d31ee64186,d4e936cfd6,f8ef6ed24a),be9c994609, s390x comparison fix (a5f8128e39),findnullvia IndexByte (91102bf723; first attempt7365fac2dbcrossed page boundaries; GoString -28%). - Go 1.12 — Done: arm64 Equal (
de28555c0b), ppc64x asm (a0fad982b1,6994731ec2,23f7554194), linknames and trampolines (ef7ce57ac2,ad4a58e315). - Go 1.13 — Done: arm word-wise Compare/Equal, shared code, Count (
029a5af6a1,a734601bdf,14a58d65e3,3496ff1d19,3cb1e9d98a), simplifiedEqual(ca0c449a6b), wasm memchr (4a7cd9d9df). - Go 1.14 — Done: mips64x Compare (
78baea836d,a1b1ba7daf), s390x SIMD Count (b6245cef3c), riscv64 (0c703b37df), wasm tweaks (d0f10a6e68). - Go 1.15 — Done: IndexAny/LastIndexAny and Rabin-Karp move (
3af92acb9f,18a6fd44bb). - Go 1.16 — Done: riscv64 Count (
19f6422e00), mips64x Equal (4f597abe77), CBZ on arm64 (99d6e3eec2), BP-clobber fix (c6a11f0dd2),IndexStringfix (11cdbab9d4). - Go 1.17 — Done: ppc64x
Index(8009a81f7a,54af9fd9e6), ABIInternal ports (8f4c5068e0).
Slices and append: growslice, makeslice and string/slice conversions
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 (2x below 1024 old cap threshold, 1.25x above) → continues as Slice growth, string building and allocation-free bytes/strings helpers (1.18+)
Design. append and copy began as C runtime helpers: a special single-byte case in copy (20c1ec263a, 38→16 ns), make([]T,0) without allocation (94c2536e3f), a type pointer passed to makeslice for GC (864c757a1c), and fix for zero-width element growth (782464aea5). Go 1.1 special-cased small byte appends (8cfed59941, Append1Byte 484→199 ns). Go 1.3 made growth account for the size class (bace9523ee, AppendGrowString -25%), checked for cap overflow (9121e7e4df), fixed a 32-bit overflow in stringtoslicerune (e1a91c5b89) and stopped setting m->locks around allocation so an append-only program triggers GC (8afa086ce6). Go 1.4 converted slice operations to Go (cc9ec52d73). Go 1.5 used memmove for append (497970f421), added missing write barriers for the copy (a73c1cef07), fixed a nil ptr with non-zero length (0d6a0d6c3f) and merged slice/sliceStruct (ab4df700b8).
Division avoidance dominated 1.7–1.12: growslice avoiding divisions (43ed65f869, GrowSliceBytes -29%; 6b0688f742), maxSliceCap table (a4dd6ea152, MakeSlice -32%), direct mallocgc calls (0150f15a92), power-of-two shifts (e53cc7ae93), sys.PtrSize use (37dd7cd040), and finally multiplication with overflow check in makeslice (05166bf54d), makemap and growslice (3b091bf6cc, Go 1.12), replacing the table. Simplified capacity check against memory (365594ad59) and overflow protection (371a5b494a), infinite-loop fix (f3e0d14313). append clears only uncopied memory (c1e267cc73, 1.8) and, in Go 1.12, uses bulkBarrierPreWrite plus memmove instead of per-element typedmemmove (96dcc4457b, AppendGrowString 123→73 ms) and skips the write barrier on the newly allocated slice (4363c98f62). The growth rule used the old length until Go 1.16 changed it to old capacity so append(s,0,0) and append(append(s,0),0) agree (2333c6299f). Compiler: avoid a spill in the append fast path (6b33b0e98e); race instrumentation (3f7f030c59, 089e482b3d). reflect.Append landed in 2010 (8b64cd9c5e).
Timeline.
- Go 1.0 — Done: 2009–2011: copy single-byte case (
20c1ec263a), typedmakeslice(864c757a1c),make([]T,0)allocation-free (94c2536e3f),SliceHeadermade public and fixed to use int (22ea3f9e49), gc-friendly SliceHeader (4e7aac5413). - Go 1.1 — Done: small appends, zero-width growth, race instrumentation (
8cfed59941,782464aea5,3f7f030c59). - Go 1.3 — Done: size-class growth, cap overflow check, 32-bit conversion overflow,
m->locks(bace9523ee,9121e7e4df,e1a91c5b89,8afa086ce6). - Go 1.4 — Done: Go port (
cc9ec52d73), correct malloc size (7bcbdbd904). - Go 1.5 — Done: memmove append, barriers (
497970f421,a73c1cef07,0d6a0d6c3f,ab4df700b8,537ddc9456zero-widthslicecopy). - Go 1.7 — Done: division avoidance (
43ed65f869,6b0688f742,a4dd6ea152,ad7448fe98,0150f15a92,44532f1a9d,6f3f02f80dzeroing tmpbuf). - Go 1.8 — Done: append clears only uncopied memory (
c1e267cc73). - Go 1.10 — Done: simplified capacity check, overflow, infinite loop fixes (
365594ad59,371a5b494a,f3e0d14313). - Go 1.11 — Done: power-of-two special case and PtrSize (
e53cc7ae93,37dd7cd040),gobyteswithout clearing (dfb0e4f6c7),_MaxMem→maxAlloc(d7691d055a). - Go 1.12 — Done: bulk barrier in growslice, no barrier on new slice, overflow-checked multiplication (
96dcc4457b,4363c98f62,05166bf54d,e85b8db604,3b091bf6cc). - Go 1.15 — Done: no bulkBarrierPreWrite when dst length is zero (
6ffca22602), race reorder (089e482b3d). - Go 1.16 — Done: growth regime decided by old capacity (
2333c6299f). - Go 1.17 — Done: slice→array pointer conversion in reflect (
760d3b2a16).
Strings: concatenation, conversion and printing helpers
Span: Go 1.0 → Go 1.17 · Status at go1.17: still the design at go1.17 → continues as Slice growth, string building and allocation-free bytes/strings helpers (1.18+)
Design. Early string runtime (string.c) added NUL on allocation for low-level getenv (1a0a6f9d50), later removed again: strings are not zero-terminated and “no code must rely on zero termination” (9fa9613e0b, 1.3, reduces allocs by 1% on json). maxstring tracking caused a data race (f9f21aa1fb) and was removed in Go 1.8 (252093f120). Go 1.1 avoided allocation for "" + x + "" (8744d35dd3, fixed by 71643b2fdb). Go 1.5 stopped copying []byte for string concatenation and comparison when the string is not retained (205ae07cd3, 69cd91a598). Conversions: slicebytetostring optimized with inlined rawstringtmp and memmove (23be728950, -27%), single-byte strings use a static table (9372e3f5ef, bda42a7a78 staticuint64s), intstring uses static bytes (2f2e8f9c81), stack buffers not exposed (4d02b12417), decoderune for range over strings speeds non-ASCII decoding by up to 11% (d295174030, Go 1.8), minor spill elimination (880c967ccd). concatstring{2..5} made consistent with compiler use (35a8bbc9ea). Printing: print(float) (ac5a887d5d, ce0f1dee6f), -Inf (69879f0414), signed zero (f574726f16), printing numeric panic values (0d33dc3105, 972df38445), and the print implementation moved to Go (721c8735df, 597b266eaf, ef64d9ffcc).
Timeline.
- Go 1.0 — Done: NUL-terminated allocation (
1a0a6f9d50), float print format (ac5a887d5d),rangeUTF-8 fix (a18208fb60),maxstringrace (f9f21aa1fb). - Go 1.1 — Done:
"" + x + ""(8744d35dd3,71643b2fdb),slicebytetostringrace instrumentation (4a524311f4). - Go 1.2 — Done: correct types for
maxstring/concatstring(98a80b95b4,249f807c39). - Go 1.3 — Done: no zero termination (
9fa9613e0b), signed-zero printing (f574726f16). - Go 1.4 — Done: print in Go (
721c8735df,597b266eaf,ef64d9ffcc,d8cbbe6802),gostringnocopyupdates maxstring (bcd36e8857), concatstring panic usesgothrow(fa5f86281e). - Go 1.5 — Done: no copy for concat/compare (
205ae07cd3,69cd91a598). - Go 1.6 — Done: string files merged and
stringStructOf(881b0e7880,d4a7ea1b71), stack buffer fix (4d02b12417). - Go 1.8 — Done:
decoderuneand string/rune optimizations (d295174030,880c967ccd),maxstringremoved (252093f120). - Go 1.9 — Done:
slicebytetostringspeedup (23be728950), numeric panic output (0d33dc3105). - Go 1.11 — Done: length-1 string and
intstringstatics (9372e3f5ef,2f2e8f9c81),findnull(91102bf723). - Go 1.15 — Done:
staticuint64sreplacesstaticbytes(bda42a7a78), panic value printing (972df38445), nostring(int)in runtime (6052838bc3). - Go 1.17 — Done: consistent
concatstringentry points (35a8bbc9ea).
Soft-float, 64-bit division and complex arithmetic helpers
Span: Go 1.1 → Go 1.15 · Status at go1.17: still the design at go1.17 (Go implementations in softfloat64.go, vlop, complex.go) → no continuation noted
Design. ARM (GOARM=5) floating point is emulated by an interpreter in the runtime: _sfloat runs after each instruction sequence and decodes FP instructions from the code stream. Go 1.1 added conditional-execution support (fb3a1b6821); Go 1.2 disabled preemption during software FP routines because the registers are kept per-M (e03dd07981), recorded _sfloat frame size and no arguments (8166b2da19, 6a13897bc6) and fixed arm preemption (cba880e04a, f4f2cf16b0); sqrt copy for softfloat (ee7bb07a53, Go 1.3). The interpreter was converted from C to Go in the dev.cc work (c81d248eca, 2ceca80e3f, Go 1.5), with nil-pointer handling (4b3906fec3), a fix for an SQRT register corruption (102436e800), add r11, pc, r11 (40af15f28e) and MININT conversion fix in the compiler (6db98a3c51). The old ARM softfloat code was deleted in Go 1.11 (f83e421268, -1065 lines) after the compiler gained new softfloat support for GOARM=5. MIPS softfloat routines were added for GOMIPS=softfloat (ac987df87c, Go 1.10), and Go 1.15 made soft-float results use quiet NaNs (d8bf079751).
Integer helpers: Go 1.1 prepared 64-bit int in the runtime with intgo typedefs (0b08c9483f), made ARM div/mod ~3.7x faster (3e3fa7b5f1, 281→75 ns), and Go 1.2 fixed uint64 division on 386 (70138a2108) and ARM div-by-zero tracebacks (b2794a1c2e). Go 1.4 implemented 64-bit division in Go (d11bb3b177) and converted float↔︎int64 conversions (32b3a48ea8, 0eaea6010a). Go 1.9 restored ARM div/mod stubs (79d05e75ca). Complex division: inlined float helpers (0157c72d13, 1.1; ARM -70%, 386 mixed), converted to Go (c403af83a6), made C99-compatible with new isNaN/isFinite/isInf/abs/copysign helpers (16200c7333, Go 1.9), and the NaN constant changed to match gcc’s for VFPv2 (6e9506a7b4).
Timeline.
- Go 1.1 — Done: conditional soft-float (
fb3a1b6821), ARM div/mod (3e3fa7b5f1), complex division speedups (0157c72d13), NaN choice (6e9506a7b4),intgo(0b08c9483f), - Go 1.2 — Done: no preemption in softfloat, argument/frame records (
e03dd07981,8166b2da19,6a13897bc6,cba880e04a), 386 uint64 division (70138a2108),timedivavoiding_vasop(c7d5c438a2). - Go 1.3 — Done: sqrt copy (
ee7bb07a53), cross-function jump removal invlop_arm.sfor the new linker (7c17982f72). - Go 1.4 — Done: 64-bit division, complex division and conversions in Go (
d11bb3b177,c403af83a6,32b3a48ea8,0eaea6010a), nil pointer in softfloat (4b3906fec3). - Go 1.5 — Done: softfloat in Go (
c81d248eca,2ceca80e3f), SQRT register fix (102436e800). - Go 1.6 — Done: “add r11, pc, r11” (
40af15f28e), mips64 support in reflect and runtime asm (559fb85af6,424738e43e). - Go 1.7 — Done: arm5 FP stores (
9cb79e9536), MININT softfloat fix (6db98a3c51). - Go 1.9 — Done: C99 complex division (
16200c7333), ARM div/mod stubs (79d05e75ca). - Go 1.10 — Done: MIPS soft-float routines (
ac987df87c). - Go 1.11 — Done: old ARM softfloat removed (
f83e421268), nacl/arm udiv (5a43a271e8). - Go 1.15 — Done: quiet NaNs in softfloat (
d8bf079751).
C-to-Go conversion of runtime data-structure code
Span: Go 1.4 → Go 1.5 · Status at go1.17: completed in Go 1.5; the runtime is Go with assembly stubs → no continuation noted
- design doc dev.cc branch plan cited in this thread's commits
Design. Go 1.4 and Go 1.5 removed C from package runtime, file by file, as part of the dev.cc plan. In this area: maps (0c6b55e76b), slices (cc9ec52d73), hash and equality (a2a9768414, 7aa4e5ac5f), interfaces (483cb61921, b53b47f5ac), print (721c8735df, 597b266eaf), 64-bit division and float conversion (d11bb3b177, 32b3a48ea8, 0eaea6010a), complex division (c403af83a6). Earlier, goc2c was used as much as possible (67c83db60d, 1.3) to avoid architecture-dependent argument layouts and avoid maintaining stack-argument pragmas. The mechanical dev.cc commits converted header files (fee9e47559) and basic library routines (0c3c2c1724). Calling-convention helpers were added: throwgo (ec5d7ba95c), funcPC (0e07f1c99f), reflect trampolines moved into the thunk file (6baf173b67).
Timeline.
- Go 1.3 — Done: goc2c (
67c83db60d), errorCString as struct (7e639c0229). - Go 1.4 — Done: conversions listed above (
0c6b55e76b,cc9ec52d73,a2a9768414,7aa4e5ac5f,483cb61921,b53b47f5ac,68f91aea57,d11bb3b177),errorCStringadjusted to avoid allocation (48452a276d). - Go 1.5 — Done: dev.cc conversions (
fee9e47559,0c3c2c1724,c81d248eca),noasm.gocmove (fb57459b48), arm assembly regularized (69ddb7a408).
internal/reflectlite and checkptr
Span: Go 1.13 → Go 1.17 · Status at go1.17: still the design at go1.17
- proposal #22218 proposal: add GOEXPERIMENT=checkptr referenced by 3 of this thread's commits
- discussion Dynamic unsafe pointer checking available at master golang-dev · linked from #34964
Design. internal/reflectlite is a small copy of the reflect subset needed by errors (assignability and setting for As), added to avoid importing full reflect (9650726e79, 2735 lines), with context then dropping its reflect and fmt dependencies (88adc33827, 724a86fced) and sort using it too (39a51a4b0d). Go 1.14 introduced pointer-conversion checks under -d=checkptr: unsafe conversions from smaller to larger types are detected (dc72a2f95f), reflect and testing/quick were fixed to pass it (5eec0a91ea, 5a5854c2d1, 9989221ac1), messages improved (daeb5efb20, 1f231d74f6 in Go 1.15), failures use throw instead of panic (56d6b87972), only check up to ptrdata (08dd4ad7e3, Go 1.15), no alignment error for pointer-free base types (f4ddc00345), and Go 1.17 does not crash on nil pointers (b39e0f461c).
Timeline.
- Go 1.13 — Done: reflectlite (
9650726e79,39a51a4b0d), context deps (88adc33827,724a86fced). - Go 1.14 — Done: checkptr conversions (
dc72a2f95f,5eec0a91ea,5a5854c2d1,9989221ac1,daeb5efb20,56d6b87972). - Go 1.15 — Done:
f4ddc00345,08dd4ad7e3,1f231d74f6. - Go 1.16 — Done: reflectlite error includes Kind (
c19725016d). - Go 1.17 — Done: nil pointers in checkptrAlignment (
b39e0f461c).
Changes in this slice that are not part of a thread are listed on the release pages: 1.0 · 1.1 · 1.2 · 1.3 · 1.4 · 1.5 · 1.6 · 1.7 · 1.8 · 1.9 · 1.10 · 1.11 · 1.12 · 1.13 · 1.14 · 1.15 · 1.16 · 1.17
Runtime: maps, built-in data structures, hashing, randomness and type metadata (Go 1.18–1.28)
Reader: Claude Sonnet. Cites 502 of 711 commits; 78 small fixes are uncited. Read commit messages only (long ones truncated); no diffs were read.
Swiss-table maps (GOEXPERIMENT=swissmap → only implementation)
Span: Go 1.24 → Go 1.28 · Status at HEAD: only map implementation since the experiment was deleted in 1.26 (2ae059ccaf); GOEXPERIMENT=mapsplitgroup (KKKKVVVV layout) still an opt-in experiment; micro-tuning ongoing in 1.27/1.28
Paths not taken: Hash-map alternatives before Swiss tables (superseded)
Earlier history: Hash map core: from the first hashmap.c to the 8-entry bucket design (Go 1.0 → Go 1.17)
- tracking issue #54766 runtime: use SwissTable referenced by 26 of this thread's commits
Design. The classic bucket-chained hmap (8-slot buckets, overflow chains, incremental evacuation) was replaced by a Swiss table in the new package internal/runtime/maps, modelled on Abseil and on github.com/cockroachdb/swiss (0733682e5f). A map is a directory of tables (extendible hashing, d94b7a1876: “splits a swisstable map into many swisstables. This keeps grow operations small”, so there is no incremental evacuation any more; a table is rehashed/split in one go and its size is capped). Each table is an array of groups: a 64-bit control word (one byte per slot: H2 = 7 hash bits, or empty / deleted-tombstone) plus 8 slots. Lookup probes group by group (probeSeq), computes matchH2 on the control word to get candidate slots, compares keys, and stops at the first group containing an empty slot. Deletion writes a tombstone unless the group has an empty slot. Small maps (<=8 entries) skip directory and table: the map points directly at one group (77e3d8cf13) and lookups scan the group linearly (808da68c1c), later with SIMD/SWAR match instead of a byte scan (bfc209518e, ~1.7x faster for unpredictable keys, from ~25ns to ~14ns). The ABI (internal/abi/map_swiss.go) and compiler side (cmd/compile/internal/reflectdata/map_swiss.go) were forked from the old map from day one (057b703407, 4f7dc282c4) so both could coexist behind the experiment; large keys/elems are stored indirectly using the same heuristics as the old map (b5fec2cf54), concurrent-write detection (fatal("concurrent map writes")) reuses the old design (2220fd3636). Hot-path work was done by the compiler too: on amd64 matchH2/matchEmpty are intrinsified with SSE (dce30a1920; still 8-slot groups to avoid changing memory layout; 0-3% faster), and the specialised fast32/fast64/faststr variants (f782e16162) were later devirtualised for hashing (252a8adbc0). Everything else in the thread is operational polish: tombstone reuse on insert (efa43c57b1), clear erases tombstones (4e63ae46e0), pruning tombstones before growing (05ed8a00e0), proper capacity-hint handling (3b424cfa9d: no allocation at hint 0; initial capacity sized to avoid growth on average).
Timeline.
- Go 1.24 — Done: user-visible
GOEXPERIMENT=swissmapcreated with forked files (057b703407), ABI/compiler integration split into old and swiss variants (4f7dc282c4), fast variants initially disabled (1985c0ccf9), initial table without incremental growth (0733682e5f), wired into the runtime enough to pass all.bash (c39bc22c14; missing: race/concurrent-write detection, stack-allocated maps, fast variants, indirect key/elem, all added later in the cycle:2220fd3636,b5fec2cf54,f782e16162). Extendible hashing (d94b7a1876), small-map direct group (77e3d8cf13),mapaccess1/mapassignmoved fromruntimetointernal/runtime/maps(0564fa6820), capacity hints (3b424cfa9d), shift/modulo optimisations (bb46b754be,cd54b9bae9), deleted-slot reuse (efa43c57b1), big-endian support (3352db152b), no re-copy of indirect key/elem on grow (d4b0bd28ee), single hash on delete (4c8ab993cd), int-key small-map lookups (d51f92e737), simpler small-group lookup that stops once no filled slots remain (a867e5e5a6, -11% geomean), constant elem offset for int64/string keys (c14fc500c7,253a9933d1), one fewer hot-path load (63f762bcde), hash a copy of the key rather than the key (d13e6d0b08). Iteration: Iter stores the group acrossNextcalls (aefb173b0a,7b3ac0ca5d) and uses a hybrid single-slot-then-group-match strategy (7c40544441: low-load maps had regressed +60% vs old maps; testing a full group match on every call regressed mostly-full maps, hence the hybrid). Small-map lookups with long string keys do a quick equality prefilter over all slots and only hash if more than one passes (04807d3acf);returnafterfatalkeeps the key in a register in the probe loop (43725ba283).reflectgot swissGroupSizesupport (02df9bbdbd) and droppedmapiterkey/mapiterelem(7ba074fe43). Compatibility:mapiterinit/mapiternextare linknamed by third parties with the oldhiterlayout, so newmapIterStart/mapIterNextwere introduced for the toolchain and the old names became a compat layer (78e6f2a1c8,8071f2a169). Left open: default on but GOEXPERIMENT still exists; nothing in the 1.24 messages noting concrete unfinished items besides the “single table without directory” idea in77e3d8cf13(“left for later”). - Go 1.25 — Done: clear also erases tombstones (
4e63ae46e0), prune tombstones before growing (05ed8a00e0, fixes #70886, pathological insert/delete workloads), faster small-map lookups (bfc209518e), empty/zero map lookups inline better (8a85a2e70a, -12.75% geomean), race PC fix (14b15a2bea), fastermaps.Clone(a645bc5eb9). Left open: thenoswissmapfallback still existed (“noswissmap is going away soon”,252c939445). - Go 1.26 — Done: user-visible
GOEXPERIMENT=swissmapremoved, ~6,000 lines of old map code deleted,map_swiss.gorenamed tomap.go(2ae059ccaf,82a1921c3b);Clearonly checks “any full slot” rather than computing the bitset (8098b99547) and only conditionally clears groups when sparse (de5d7eccb9); hoistedh2(hash)out of loops (b096ddb9ea); 32-bit hashes on wasm because 64x64->128 multiplies are not intrinsics there (a26f860fa4, -26% Hash geomean). Left open: nothing noted. - Go 1.27 — Done: user-visible
GOEXPERIMENT=mapsplitgroupadded, off by default: groups becomectrl; keys [8]K; elems [8]Vsomap[K]struct{}no longer pays a padded elem per slot (5560073398; trade-off stated: better key cache behaviour, elem no longer at a fixed offset from its key). Loweringmapaccess1*tomapaccess2*at walk time to share one implementation (62d08234b7); undone in the same cycle becauseOINDEXMAPis used in expression context and late replacement generated invalid code (2a4fbc3f24), leavingmapaccess1as a wrapper aroundmapaccess2. Devirtualised hashing in specialised maps (252a8adbc0), pointer-key fast variants were missingpruneTombstones(03dc8c482f,3cf84263ec),ctrlGroupMatchH2on arm64 rewritten to avoid cross-byte borrow false positives (f133609b75), small full maps only grow when inserting a new key (8042aaf03c), correct const for indirect elem flag (51a8f213cd). Left open:mapsplitgroupnot default; hashing not yet inlined into map functions (252a8adbc0says “opens opportunity for further improvement by inlining hashing calls in the future”). - Go 1.28 — Done:
mapaccess2inlined intomapaccess1by raising the inline budget, because the wrapper cost showed in synthetic benchmarks (defa1f0155; ideal of inliningmapaccess1at the call site judged hard since calls are introduced in walk, after inlining). Small-key hashing switched from AES to wyhash belowMinAeshashSize(c9e4a2aa07,b751a977cb; see hashing thread). Left open: latency-vs-throughput cutoff for arm64 (go.dev/issue/80970) noted inb751a977cb.
Map hashing and hash functions (memhash, aeshash, fallback hash)
Span: Go 1.23 → Go 1.28 · Status at HEAD: hashing code for maps lives in internal/runtime/maps (moved from runtime), hybrid wyhash/AES keyed by MinAeshashSize, interface hash folds the dynamic type
Earlier history: Hash functions and algorithm tables: memhash, AES hash, fallback hash (Go 1.0 → Go 1.17)
Design. Map hashing is memhash{32,64}, strhash, and memhash over arbitrary memory, selected between AES-NI/ARMv8-crypto implementations and a portable multiply-mix fallback (hash64.go/hash32.go), seeded per process from hashkey/aeskeysched. Over this period the work was (1) closing weaknesses in the fallback, (2) moving ownership of the hash code into internal/runtime/maps so it can be inlined/intrinsified, (3) tuning when AES is worth using.
Timeline.
- Go 1.23 — Done: the 64-bit fallback hash
mix()first argument was attacker controllable (xor of input with constants) so zeroing the multiplier produced many collisions; it is now xored with the process seed (ef2f3391da), then constants xors removed/odd-forcing dropped (c260de24b6,35c619398b).hashkeywas deterministic because it was initialised beforerandwas seeded; now usesbootstrapRand(1a3682b4c1).memhash{32,64}lost their external linkname (2e9ed44d391.24, landed in the 1.24 cycle). Left open: nothing noted. - Go 1.26 — Done: 32-bit hash on wasm (
a26f860fa4). Left open: nothing noted. - Go 1.27 — Done: all assembly hash functions (386/amd64/arm64, aes and noaes) moved from
runtimetointernal/runtime/mapsas “ground work” for rewriting with SIMD intrinsics and inlining into map functions (8594bf4621, 34 files);MemHash{32,64}rewritten withsimd/archsimdintrinsics underGOEXPERIMENT=simd(2403e594a5, -8.67% on one geomean, others within noise; MapAccessZero/Empty swings attributed to basic-block alignment); keys passed by value toMemHash{32,64}/StrHashto avoid address-taken keys and reloads, andstrHashAESfolded intomemHashAES(05f75fb9e8). Left open: hash inlining into map ops still future work. - Go 1.28 — Done:
UseAeshashreplaced by a tunableMinAeshashSize: very small keys use wyhash because AES SIMD “does not like being used for one-offs”: cutoff 8 bytes on Zen 4 (MapGroupBy, HashInt64Speed -10%, MapAccessMiss/int64/len=65536 -10.9%) (c9e4a2aa07), 16 bytes on Apple M1 Max (b751a977cb: Hash5 -41%, MapGroupBy/int64 -65%). Interface hashing now folds the dynamic type’s compiler typehash in soint64(0)anduint64(0)in interfaces no longer collide (40e7643c46). Left open: cutoffs for other microarchitectures (Intel steppings) are said to be easy follow-ups; latency cutoff of 192 on arm64 unexplored (issue 80970).
Compiler-generated equality/hash functions and type descriptor emission (reflectdata)
Span: Go 1.18 → Go 1.28 · Status at HEAD: one equality/hash function per “signature”; large types call subroutines; alg and noalg descriptors share one symbol
Earlier history: Compiler-generated equality and hash functions (Go 1.0 → Go 1.17)
Design. cmd/compile/internal/reflectdata writes runtime._type descriptors, itabs, uncommon-type method tables, and the ==/hash functions referenced from them (alg.go). Struct comparison logic was unified into cmd/compile/internal/compare (3c29aca436) so walk’s == lowering and generated eq functions share the “regular memory” analysis, and array equality loops are unrolled (1a955bcdb7: -18.6% geomean). In 1.21 equal/hash generation was moved to the walk phase and factored out (ea3054aecc, 864b54e2c3, 4636c57fe1). The 1.27 redesign (30dff416e4): instead of one equality function per type, generate one per signature, a compact summary of the comparisons needed. struct{i int32; j uint32; s string; e error} has signature M8SI (8 bytes regular memory, string, non-empty interface), so many types share one function. Generated equality functions in the go binary dropped from 634 to 286, binary ~1% smaller. Hash generation was moved onto the same signatures (0da8979210), with parsing simplifications (59ed013621, 1bd5dbfc41). In 1.28 large types call their eq/hash as a subroutine instead of inlining the body (16516099d7), and a type and its noalg variant (the descriptor without algorithms, used for hidden pointer/array types) now share one symbol; the ordinary descriptor wins and a noalg descriptor is “upgraded” in place (CL 829124 per 8fc7e76411), which needed follow-up fixes: restoring TFlagRegularMemory and not folding TFlag loads from noalg types (8fc7e76411), emission order alg before noalg (925ef8c02c), keeping the typelink on upgrade (f3e4f788ed, via setAlgAttrs). Compile time work: writeType reuses the already-looked-up type linksym (7ea76879d2, -5% wall on typescript-go/ast). SSA now folds loads of abi.Type.{Size_,PtrBytes,Kind_,Hash} and PtrType.Elem from statically known descriptors (5b218461f9; counts when compiling std/cmd: 75 Kind_, 50 PtrType.Elem, 14 Hash).
Timeline.
- Go 1.18 — Done: generics support in type metadata: always generate
(*T).Mwrappers for instantiated methods (4ed6317e73), crawl embeddable types so all methods are re-exported (132ea56d29, allowsmethodWrapperto always inline),Type.LinkStringfor map keys in typecheck/order (ed647b16d0), avoid redundant wrappers for anonymous types from imported packages in unified IR (7ab8754029, protobuf-heavy builds blew build-cache limits), shape types in their own package (b730a26729), no methods on shape types (48cf96c256), dictionaries reuse global node (8eeb1bff1d), type descriptors always dupok (79c5240799), prevent duplicate work inWriteRuntimeTypes(91e2e3b903),anydefinition emission when compiling runtime (3d7cb23e3d), unsatisfiable type assertions in instantiations (cf1ec17360). Left open: unified IR flagged as in-progress. - Go 1.19 — Done: array-equal loop unrolling (
1a955bcdb7), instantiating types excluded from type hash (d3672054fb),dupokadjustments for shape/non-generic types (884530b374,909a7a3213), incomplete itab always writesfun[0](6f5590edf6), struct comparison unification (3c29aca436), embedded bit moved from field offset to name bytes, leaving a full uintptr for offsets (e1e66a03a6), sort named types before unnamed for plugin signature (5b4fafd5db). Left open: nothing noted. - Go 1.20 — Done: Unified IR starts setting
RTypefields so reflectdata no longer re-derives runtime types from IR (7368647ac6,f70775ff22,2cf632cd57), fewer basic types emitted in every compilation (cb5534c1c6), no content-addressable symbol for generic iface method names (8dc7710fae). Left open: “subsequent CLs will handle the remaining more complex cases” (7368647ac6). - Go 1.21 — Done: equal/hash generation moved to walk and factored (
ea3054aecc,864b54e2c3,4636c57fe1,25f5d9d4a2; one revert/reland95be547e0a/eee2697c38),AfterGlobalEscapeAnalysisremoved (8ffcd3da93). Left open: nothing noted. - Go 1.22 — Done: reflectdata uses
internal/abitypes directly (7fcc626b57) and descriptor generation reorganised aroundrttype(2d9e8ded94). Left open: nothing noted. - Go 1.23 — Done: itabs generated via the
rttypemechanism (977803e796), compiler usesabi.MapMaxKeyBytesetc. (4605ce2d34),MaxPtrmaskBytesmerged intointernal/abi(ff4e45fbc2). Left open: nothing noted. - Go 1.24 — Done: GC programs removed from type descriptors; the runtime builds the pointer bitmap on demand (
45869f5931, -110 net lines: reflectdata, linkerdata.go/decodesym.go,abi.Type,reflect.type); abi.NoEscape treated as cheap call (b6efbd4efc). Left open:MaxPtrmaskBytesstayed at 2048 (revisited in 1.27). - Go 1.26 — Done: ptrmask no longer cleared in
fillptrmask(e0807ba470), type-load folding (5b218461f9),TypeFormade inlineable (dc960d0bfe; also forcesTypeLinksyminwriteTypefor non-shape types so later lookups see the symbol). Left open: “TFlag … decent help … if statically known” (5b218461f9), implemented later (the “TFlag folding” CL 802360 cited in8fc7e764111.28). - Go 1.27 — Done: signature-based equality (
30dff416e4,6eec9bcdb2), signature-based hash (0da8979210), type descriptor alignment set to 8 (a430bc1bd7: cmd/go -88K),MaxPtrmaskBytesback to 16 since GC programs are gone (a409055b7c; tailscaled: types needing runtime-computed masks 6→49, ~100 bytes saved), descriptor size constants moved tointernal/abi(d440b8b485), zero-size array element crash fix (cf0c42c2ca), linker handling ofruntime.typesleading offset based on size, not GOOS (20bc194972), AIX user-created GCData adjustments (bd1b41eb81). Left open: “Maybe a TODO for the future”: only generated comparison functions use signatures, not the code generated for==itself (30dff416e4). - Go 1.28 — Done: subroutine-based eq/hash for large types (
16516099d7), noalg/alg dedup and its fixes (8fc7e76411,925ef8c02c,f3e4f788ed,4400f9ad36),writeTypesymbol reuse (7ea76879d2), cleanup ofcurfuncnaming in reflectdata (d5f5390d0d). Left open: signature names remain large (“The signatures themselves (and their names) are still pretty big”,16516099d7).
internal/abi type layout unification and type metadata (rtype → abi.Type)
Span: Go 1.21 → Go 1.27 · Status at HEAD: abi.Type is the single type-descriptor definition used by compiler, linker, runtime, reflect and reflectlite; direct/indirect interface flag lives in TFlag
Earlier history: Runtime type descriptors: layout, shrinking and name encoding (Go 1.0 → Go 1.17)
Design. Before 1.21 the layout of type descriptors was duplicated four times (compiler, runtime._type, reflect.rtype, internal/reflectlite) and had to be kept in sync by hand. The effort, driven by David Chase, collapsed this into internal/abi: constants for map layout (e22bd2348c), then the common abi.Type plus Method, Imethod, UncommonType, ArrayType, ChanType, FuncType, StructType (bdc6ae579a refactors “(basic) type struct into one definition” touching 46 files; d9a61be73f, a851511947, 5deeca38c2, acba24ff8c, 639957eb66, b768a1fbb5). Then runtime _type is redefined as abi.Type with an rtype wrapper for methods (a2838ec5f2, 2e93fe0a9f), reflect replaces nearly all *rtype with *abi.Type (6cf7602b71), internal/reflectlite is commoned up (3ee12d5702). A reflect change making rtype itself (not *rtype) implement Type was reverted (6d2309b576). The reorganisation also added bootstrap-with-gccgo support (6691f438c3). Later refinements: itabs use abi.ITab in reflect and drop a blank field (f2db96cd6c, cdd0ddaf45), abi.EmptyInterface, TypeOf, NoEscape (4742c52e10), abi.Kind* used throughout (daaf1f2220), go122 flag removed (3902e9ef4e), arch-family-based constants (f69703d389), bounds check constants (741a19ab41).
Timeline.
- Go 1.21 — Done: above;
bdc6ae579a,a2838ec5f2,6cf7602b71,3ee12d5702. Left open: “the new files in internal/abi are copied from the end of the stack; ultimately this will all end up being used” (bdc6ae579a), i.e. the rtype receiver change was reverted (6d2309b576). - Go 1.22 — Done: compiler uses internal/abi types (
7fcc626b57), map layout read from abi (cc47df017duses real type size in map keys/values). Left open: nothing noted. - Go 1.23 — Done:
abi.ITabin reflect (f2db96cd6c),EmptyInterface/TypeOf/NoEscape(4742c52e10),zeroValmoved to internal/abi and then reverted back to runtime (2f5b420fb5,4c589d93ad),ifaceIndirremoved then restored “for now” (55a06f7920,2064413b54). Left open: nothing noted. - Go 1.24 — Done: GC programs removed from
abi.Typehandling (45869f5931). Left open: nothing noted. - Go 1.26 — Done: the direct/indirect-interface bit moved from the high bit of
KindtoTFlag, so kind access needs no mask;IfaceIndir→!IsDirectIface; gocore and delve had to be updated and the commit lists “TODO: any other tools need updating?” (f7d167fe71). Left open: debugger/tool updates explicitly open. - Go 1.27 — Done: type descriptor sizes in
internal/abi(d440b8b485),MaxPtrmaskBytes=16(a409055b7c),reflect.typelinksreturns types, not offsets, via new runtime accessor; old function kept for linknamers (87fae3622d). Left open: nothing noted.
Interface conversion: itabs, type switch and assertion caches
Span: Go 1.19 → Go 1.28 · Status at HEAD: per-call-site caches in front of interface type switches/assertions; itabs generated via rttype
Paths not taken: GOEXPERIMENT=largeiface (never landed)
Earlier history: Interface values, itabs and type assertions (Go 1.0 → Go 1.17)
- talk Interface Internals. GopherCon 2024 listed in Go: A Documentary
Design. Type switches over interface cases and assertions to interface types called into the runtime (typeAssert, interfaceSwitch) each time. In 1.22 the compiler began emitting a per-site cache, “a hash table of a sample of all the concrete types that have been switched on at that source location”, recording the matched case number and the resulting itab for each concrete input type (39263f34a3); the same cache fronts assertions (cbcf8efa5f). Caches stay small: only two in all.bash exceed 100 entries (issue29264.go 101, issue29312.go 254, both at fmt.(*pp).handleMethods). Runtime itab handling: unnecessary nil checking removed (e7d582b55d), itab not re-initialised during missing-function lookup (2589a89468), unsafe.Slice in itabInit (9777ceceec), and no write barrier for code pointers in itabInit because on wasm a PC (function index << 16) can look like a heap pointer, crashing GC (9d2ed75ac0, itabInit marked //go:nowritebarrier). Plugin itabs no longer added twice (2d04c9d8b6).
Timeline.
- Go 1.19 — Done: incomplete itab always writes
fun[0](6f5590edf6). Left open: nothing noted. - Go 1.22 — Done: caches for interface type switches (
39263f34a3) and type asserts (cbcf8efa5f); itab init nil-check cleanup (e7d582b55d). Left open: nothing noted. - Go 1.23 — Done: compiler generates itabs with
rttype(977803e796),2589a89468. Left open: nothing noted. - Go 1.27/1.28 — Done: 1.27
9777ceceec; 1.289d2ed75ac0(wasm GC crash),2d04c9d8b6(plugin duplicates). Left open: nothing noted.
reflect implementation: performance, allocation and escape behaviour
Span: Go 1.18 → Go 1.28 · Status at HEAD: reflect.Value is stack-allocatable, many accessors inlineable, method Values eagerly materialised
Earlier history: reflect redesign: from string parsing to Type/Value structs (Go 1.0 → Go 1.4) · reflect dynamic types: ArrayOf, ChanOf, MapOf, SliceOf, FuncOf, StructOf (Go 1.1 → Go 1.14) · reflect performance, caches and API additions (Go 1.4 → Go 1.17)
Design. The recurring theme is making reflect cheaper for encoding/json, text/template, etc. Strategies: reuse compiler/runtime specialisations (string-key map access via mapaccess_faststr, 23832ba2e2), avoid allocation (DeepEqual fast paths for simple kinds, a0c409cbc8: Tailscale HostInfoEqual 25.9µs→14.4µs, allocs 548→90), make hot methods inlineable (67d6be139c: Bool, String, Bytes for []byte, Len/Cap for slices; ~10-20% slower for string/map Len, substantially faster for slices; e948c3394e, c5edd5f616, 67d6be139c in 1.19), stop Value escaping (0ac72f8b96, be4fe08b57, 3e19dc2b23, 23ca330095, d573a8a8e8), and use equality functions in IsZero (1ab6b790be, then array/struct zero-bit-pattern shortcuts 12051f7d95 -69.56% geomean, ada5c2edb4, and a pointer comparison against the shared zeroVal that the compiler now uses for zero composites in interfaces, a8e0641d5b, -51.7% geomean). MapIter was reworked repeatedly: embedded hiter allocation (1b2d794ca3), Reset, SetIterKey/SetIterValue (54cdef1f10, 8dab959a8e), avoiding stack copies of hiter seen as duffcopy in profiles (89c527007f), then dropping mapiterkey/mapiterelem to speed Next (7ba074fe43), at the cost of a possible heap escape of the map bucket in test/escape_reflect.go.
Timeline.
- Go 1.18 — Done: user-visible
MapIter.SetKey/SetValue→Value.SetIterKey/SetIterValue,MapIter.Reset,Value.UnsafePointer,Value.CanInt/CanUint/CanFloat/CanComplex,FieldByIndexErr,Pointer/PointerTokind renames (54cdef1f10,8dab959a8e,4d550727f8,04f7521b0a,33c392f72d,17910ed4ff); string-key map fast path (23832ba2e2, “over 80% of all Go map types use a string as the key”),DeepEqualfast paths (a0c409cbc8),hiterembedded inMapIter(1b2d794ca3,89c527007f), Elem/Pointer panic on bad notinheap pointers (8331f25e96,23e57e5955). Left open: nothing noted. - Go 1.19 — Done: inlineable
Value.Type,MapRange,Bool,String,Bytes,Len,Cap(e948c3394e,c5edd5f616,67d6be139c,01837ca7eb);Value.Byteson addressable arrays (a5b8b56d1d);Len/Capon pointer-to-array (892cd0b636);SliceHeader/StringHeaderdeprecation added then reverted (bc2e961cf4,e6d8b05a61); map key type check inMapIndex/SetMapIndex(11a650bb4a);StructOfoverflow panics (c22a6c3b90). Left open: nothing noted. - Go 1.20 — Done: user-visible
Value.Grow(61f0409c31),SetZero(bc1d0d8eb1),Comparable/Equal(acabf87127;Equalrewritten to avoid allocationa343f4017b, panics on two non-comparable values7ffc1e47b4), slice→array conversion (805f174e5c),FuncOfsupports >50 args (d7a3fa120d) by building func types throughStructOf(63668a5155),rtype.MethodByNameby binary search (0053ec452d),IsZerousesrtype.equal(1ab6b790be),unsafe.Stringfor names/tags (26b2c9a620),SliceHeader/StringHeaderdeprecation added (d734203e39) then notices removed because no proposal discussion (9f2951b5d1). Race onfuncTypesfixed (6d8ec89303). Left open: deprecation postponed to 1.22 per wiki rule. - Go 1.21 — Done:
Valuecan be stack-allocated (0ac72f8b96),Typedoes not escape (be4fe08b57),Value.Clear(d42c08a2be),SliceHeader/StringHeaderdeprecated again (33d8cdeedc), unusedtypedmemmovepartialremoved (df6396fc22). Left open: nothing noted. - Go 1.22 — Done: user-visible
reflect.TypeFor(42d2dfb430),PtrTodeprecated (93ec88160e),DeepEqualfor maps optimised (f024e390bb),IsZerofor arrays and structs (12051f7d95,ada5c2edb4,c700c23cef), panic on recv channel close (f2d709b769),StructOfembedded-interface support removed since it never worked (method wrappers need closure calling convention) (6d5c9f2f26), reflectliteValuestack-allocatable (d573a8a8e8). Left open: nothing noted. - Go 1.23 — Done: user-visible
reflect.SliceAt(d08a957298),Type.OverflowInt/Uint/Float/Complex(33013e8ea8), iteration methodsValue.Seq/Seq2,Type.CanSeq/CanSeq2(524a774634,b44600f83f),TypeForfaster for concrete types (2413629aa8),Comparabletrue for nil interface (e41fabd688), String kind inPointer/UnsafePointer(3d61f24835),StructOfsetsTFlagRegularMemory(99e12bee44) with an array fix (3330c69476), anonymous field name omitted from type string (2ff89341f6). Left open: nothing noted. - Go 1.24 — Done:
structType.Fieldavoids allocation (493517bb5c), swissGroupSize(02df9bbdbd),MapIterwithoutmapiterkey/elem(7ba074fe43),MapOfflag tests (7dac9898c3). Left open: nothing noted. - Go 1.25 — Done: user-visible
reflect.TypeAssert[T](2541a68a70; zero-alloc for concrete T, allocates for methods or interface T fromElem()), later fixed for nil interface (f1e6ae2f6f); fasterTypeFor(679cd8e779),IsZeropointer check (a8e0641d5b),packEfaceleaks input to result rather than heap (ef20ccc10b),runtime.AddCleanupinstead of finalizers (b4e992b6e1),Seqfixes (d93f6df0cc,194696f1d1). Left open: nothing noted. - Go 1.26 — Done: user-visible iterator equivalents
Type.Fields/Methods/Ins/Outsstyle (e8fdfeb72b);TypeAssert[I]allocates only on success (684e8d3363);TypeForfully inlineable (dc960d0bfe); zero-sized fields of direct structs fix attempted then reverted (b3388569a1,caf9fc3ccd). Left open: the reverted fix. - Go 1.27 — Done: conversions between slices of named byte/rune and string (
a0796d8af6),Select[]runtimeSelectkept on stack (094aacdb04: B/op -12.9%),&zeroVal[0]for zero-sized payloads (60dc96479a),Value.Methods/Fieldsexamples and nil-interface panic (31768104cb,891f4a8711),MapMaxElemBytescorrection (f7d886207f), reflectlite cleanups (b04e7f31c8,d8034799e0). Left open: nothing noted. - Go 1.28 — Done: method
Valuerepresentation reworked: previously (receiver, method index, special flag) with lazy conversion to a func Value onInterface/Convert, which flowed the Value to a heapmethodValueand made those operations escape their receiver; now the func Value is created eagerly, and the receiver+index are stored in themethodValueclosure soCallkeeps its fast path (creating a method Value costs an allocation, Call unchanged) (3ebc191975, withc78af9ddc8improvingValue.Setescape);MethodByNameavoids building a fullMethodviaFuncOf(1a1b710b4c, motivated by text/template, e.g. Gitea’s ~4,700ctx.Locale.Trcalls),a795dd2918; embedded reflect types extracted before use (cff61723f3);SetIterKey/Valuetemporaries without map flags (d9377b86ee). Left open: nothing noted.
Register ABI completion for reflect, memmove/memclr and bytealg across architectures
Span: Go 1.18 → Go 1.27 · Status at HEAD: register ABI unconditional on all ports; non-register fallback paths deleted per port
Earlier history: reflect.Call, MakeFunc, method values and argument-frame GC precision (Go 1.1 → Go 1.17) · Register ABI support in reflect and runtime assembly (Go 1.17) (Go 1.17 → Go 1.17)
- tracking issue #40724 cmd/compile: switch to a register-based calling convention for Go functions referenced by 9 of this thread's commits
- design doc Proposal: Register-based Go calling convention Austin Clements, 2020-08-10 · linked from #40724
Design. The register-based internal ABI landed on amd64 in 1.17; this thread is the per-architecture rollout in the data-structure primitives: reflect.Call/MakeFunc/callMethod register spill/unspill, internal/abi IntArgRegs/FloatArgRegs constants, and ABIInternal ports of memmove, memclrNoHeapPointers, memequal, cmpbody, indexbyte, Index, Count. For each architecture the pattern was: add support (experiment-gated), always enable, then delete the fallback.
Timeline.
- Go 1.18 — Done: amd64 always on (
eeb7899137, bootstrapping hard-codes amd64), non-regabi fallback removed from amd64 runtime/bytealg (d7d4f28a06); arm64 reflect MakeFunc (0c123cdf8b), bytealg and runtime hot functions (5a40fab19f,5a008a92e8), ppc64x bytealg/memmove/memclr/reflect (b8a601756a,6c3cd5d2eb,daec057602,9c43872bd8,24c2ee7b65, adds duffcopy on ppc64x),internal/abiconstants for ARM64/PPC64 (c3639918d1,9cbdc1d48f),reflectbig-endian fix (f1d8ea1da3,fb84d213a8), softfloat functions rewritten to avoid floats (aa3d54da07). Left open: arm64/ppc64 still switchable. - Go 1.19 — Done: register ABI always on for ARM64 (
8ff42d1bb1, fallback deleted9f252a0462) and PPC64 (b810a74da3,0a69c98214);RegabiArgsconditions dropped in reflect/runtime (d8bee94be2); riscv64 runtime/reflect/bytealg/memmove (8fefeabb35,b55a2fb3b0,0652274c10,be1d7388b3; reflect support reverted oncead646b33c9, re-added56400fc706/bb2b73635f1.23, float3293218ea1a7); loong64 support (2a5114f49a,96a10b9c84,d28bf6c9a2). Left open: riscv64 still gated. - Go 1.20 — Done: riscv64 always on and fallback deleted (
7e72d384d6,5d59fa143a). Left open: nothing noted. - Go 1.22/1.23 — Done: loong64 regABI in duff, bytealg, memclr/memmove, reflect (
7b26cb9540,1052d09dd7,c83b5fefab,4e6bbbe61f,3107fa99ac), always on in 1.23 (59209c4b35,ec711aaaaa). Left open: nothing noted. - Go 1.26/1.27 — Done: s390x port of memmove/memclr, bytealg,
internal/abiconstants, reflect (4529c8fba6,2fe0ba8d52,1036f6f485,2a185fae7e) in 1.26; always enabled and fallback deleted in 1.27 (6da07b9b44,3bad3fde07). Also duff device removed for arm64 and amd64 (e4291e484c,a651e2ea47). Left open: nothing noted.
memmove / memclr / memequal / internal/bytealg assembly optimisation
Span: Go 1.18 → Go 1.28 · Status at HEAD: per-architecture hand tuning continuing; arm64 SIMD memmove landed then reverted in 1.28 cycle
Earlier history: memmove, memclr, duff routines and pointer-write atomicity (Go 1.0 → Go 1.17) · internal/bytealg and string/byte comparison primitives (Go 1.2 → Go 1.17)
Design. runtime.memmove, memclrNoHeapPointers, memequal, and internal/bytealg (Compare, Equal, Index*, Count*, IndexByte*) are hand-written assembly per GOARCH. Work follows microarchitecture features: alignment (PCALIGN loops), wider vectors (AVX2 double-lane Count, NEON, LSX, RVV), ERMS REP STOSB for memclr, Power9/10 instructions, and fast paths such as memequal returning immediately for equal pointers.
Timeline.
- Go 1.19 — Done: amd64 memclr uses ERMS
REP STOSBfor 2KB..32MB (dcfe57b8c2, -8.43% geomean; 2KB threshold from Intel’s guide), AVX2-less memclr removed for GOAMD64>=3 (12acf9b0f0), pointer-size word stores in memclr (e0f99775f2), ppc64x memmove/memclr/equal/compare/index/indexbyte (9ccf5b8e86,91b9915d3f,8375b54d44,78fb1d03d3,7ca1e2aa56,5c707f5f3a), riscv64 memequal/compare (3c42ebf3e2,291bda80e5), dead code for GOAMD64 levels removed (330cffb8691.20 in 1.20 cycle, -14.86%). Left open: nothing noted. - Go 1.20 — Done: riscv64 memclr/memmove (
44ad662a49,e18d07ddc5), alignment-check correctness (fd82718e06), ppc64xPCALIGN(92c7df116e). Left open: nothing noted. - Go 1.21 — Done: Power10 memclr/memmove/equal/index/compare (
e6faa375b4,cdf77c7209,8377f2014d,56c4422770,5f1a0320b9,db32aba508), riscv64 compare unaligned 8-byte loads (261fe25c83, -11..17%), a series of riscv64 alignment fixes (a4772a1a59,bfa25c3f6c,9552a1122f,03cd8a7b0e),PCALIGNmodified to ensure alignment (e23322e2cc). Left open: nothing noted. - Go 1.22 — Done: amd64
Index/Count/IndexBytealigned hot loops with PCALIGN (dfc1437580,3393155abf,23711f8ef7,8b6e0e6e8e: two AVX2 lanes per Count loop, -5.4% geomean), arm64 Count (289b823ac9), Power10 compare/Count (b2e809bab5,756841bffa),memequalPCALIGN (11e9c8e6a1), genericLastIndexByte(890a62bf1b), Rabin-Karp helpers via generics (b46aec0765,68e52bc03c,a05a25cb19). Left open: nothing noted. - Go 1.23 — Done:
strings.Compareintrinsified throughbytealg.CompareString→runtime_cmpstring, ~28% faster, becauseslices.SortFuncneeds a three-way compare (fd999fda59); arm64memequalshort-circuits identical pointers (66b776b025: O(n)→O(1), geomean 43.5ns→2.4ns; all other arches already had it); wasm memchr result extension fix (90c6558b6a). Left open: nothing noted. - Go 1.24 — Done: loong64 memmove/memclr/Compare (
547baafa37,501b389efc,50e536daa1), arm64Equal(b915399e7e), arm64 memclr block alignment (b4712ab055), riscv64 IndexByte (606a0bd9c8). Left open: nothing noted. - Go 1.25 — Done: arm64 Count (
f2e9076764) and Index* (5b36f61356, ABIInternal), loong64 SIMD memmove/memclr/memequal/indexbyte/compare/Count (13b7c7d8d2,1763ee199d,ecdd429a3b,7bc714d601,a1c3e2f008,17b9c9f2ad), s390x memclr (a177448765), mips64x Count (1776255742), riscv64 simplified (ad39fd9486) and synthesised MIN/MAX (d88f93f720). Left open: nothing noted. - Go 1.26 — Done: RISC-V vector
indexbyte,equal,compare(3406a617d9,75ea2d05c0,d83b16fcb8), mips64x/s390x memclr (b6cf1d94dc,a8edd99479), loong64 Index (dd3abf6bc5), arm64DC ZVAmnemonic instead of WORD (e7351c03c8), benchmark for small sizes (e84ed38641). Left open: nothing noted. - Go 1.27 — Done: android/arm64 fixes to never read outside 16-byte regions (
f5479628d3,1a0e1fb616), Rabin-Karp helpers unexported (f1e016bb3a). Left open: nothing noted. - Go 1.28 — Done: arm64 indexbyte algorithm (
5afcc450ee), compare 32 bytes per iteration and NEON for large inputs (e24c7a6508,ce608609c1), small-sizememequalrestructure (916488f43a), loong64 LSX for 33..64-byte memmove (55d79d1e8a), SIMD arm64 memmove (91aa6bb390) reverted a month later (dac9fe5ea9, named-register cleanupb6c48d70caretained). Left open: arm64 SIMD memmove needs redo.
Slice growth, string building and allocation-free bytes/strings helpers
Span: Go 1.18 → Go 1.23 · Status at HEAD: smoothed growth formula in nextslicecap, bytealg.MakeNoZero used by bytes/strings
Earlier history: Slices and append: growslice, makeslice and string/slice conversions (Go 1.0 → Go 1.17) · Strings: concatenation, conversion and printing helpers (Go 1.0 → Go 1.17)
- discussion slices grow at 25% after 1024 but why 1024? golang-nuts · cited in this thread's commits
Design. growslice used 2x growth below 1024 elements and 1.25x above, which was discontinuous and non-monotonic. 2dda92ff6f (1.18) replaces it with a smooth formula starting after 256 elements: starting cap 256→2.0x, 512→1.63, 1024→1.44, 2048→1.35, 4096→1.30 (actual growth is larger due to size-class rounding), and growth is now monotonic in initial capacity; 256 chosen to match the total reallocation count for building very large slices. nextslicecap (1d84b02b22) factors the computation out so specialised growslice variants share it, after a micro-tuning of the loop (c56f463412). For strings, bytealg.MakeNoZero allocates a []byte without zeroing, used where the caller overwrites every byte (bytes.Join/Repeat/ToUpper/ToLower, strings.Builder.Grow, transitively strings.Join/Map/Repeat/Replace…) (132fae93b7). Builder.Grow later rounds to size classes (968b71bce4).
Timeline.
- Go 1.18 — Done: smoother growth (
2dda92ff6f),unsafe.Slicebounds check and up-to-end-of-address-space (4efa216c9d,40f82f8a09). Left open: nothing noted. - Go 1.20 — Done: panic when
unsafe.Sliceparam is nil and length >0 (52d0667e6b). Left open: nothing noted. - Go 1.21 — Done:
MakeNoZero(132fae93b7),unsafe.Sliceusable from nowritebarrierrec (da384766a0). Left open: nothing noted. - Go 1.22 — Done: growslice loop optimisation (
c56f463412),nextslicecap(1d84b02b22). Left open: nothing noted. - Go 1.23 — Done:
Builder.Growsize classes (968b71bce4). Left open: nothing noted. - Go 1.27 — Done:
moveSliceNoCaprounds size down to a whole multiple of element size (1a15699a43). Left open: nothing noted.
Runtime randomness: fastrand, chacha8rand and math/rand/v2
Span: Go 1.19 → Go 1.26 · Status at HEAD: internal/chacha8rand generator with SIMD block functions on amd64, arm64, loong64, riscv64
- proposal #61716 math/rand/v2: revised API for math/rand referenced by 1 of this thread's commits
- discussion math/rand/v2: a new API for math/rand and a first v2 for std linked from #61716
Design. Runtime randomness (rand, cheaprand) is built on a ChaCha8 stream generator in internal/chacha8rand (also backing math/rand/v2.ChaCha8): buffered output blocks, generic Go block function plus assembly with 128-bit vector math. In 1.22 math/rand/v2.ChaCha8 was added “at about the same speed as PCG (25% slower on amd64, 2% faster on arm64)” (6382893890), rolled back for big-endian and riscv64 failures (82fc03f9c9) and replayed with endianness fixes (d92434935f); runtime.fastrand was kept temporarily because of third-party linknames (dca2ef2361: to be removed in 1.24). Map iterators start at a random offset using fastrand64 (01b8f5e882).
Timeline.
- Go 1.19 — Done:
fastrand64inmapiterinit(01b8f5e882). Left open: nothing noted. - Go 1.22 — Done: user-visible ChaCha8 (
6382893890, revert82fc03f9c9, replayd92434935f);fastrandback temporarily (dca2ef2361). Left open: riscv64 tests disabled because of compiler bug #64285. - Go 1.23 — Done:
internal/byteorderuse (8623c0ba95). Left open: nothing noted. - Go 1.25 — Done: loong64 assembly
block(b8ed752d6f: 40-45% faster). Left open: nothing noted. - Go 1.26 — Done: riscv64 vector implementation (
6f04a92be3), loong64 instruction mnemonics (5defe8ebb3,01c29d1f0b). Left open: nothing noted.
Concurrent hash-trie map (internal/sync.HashTrieMap)
Span: Go 1.24 → Go 1.26 · Status at HEAD: stable internal API used by unique (and later sync.Map)
Design. HashTrieMap[K,V] is a lock-free-read concurrent hash trie: nodes are indirect (children indexed by successive hash bits) or entries (key/value with overflow chain). It was moved from internal/concurrent to internal/sync “in anticipation of using HashTrieMap from the sync package” (4804d0daca). It was generalised: keys compared with normal comparison (36b5b053be), relaxed value constraints (008904aba6), zero value ready to use (e8c5e6d635), and gained the full sync.Map operation set: CompareAndSwap, Swap, Store, LoadAndDelete, Delete, Clear, Range (85fa418fd5, 28bac5640c, 872031dc10, 700c7b95ae, 083045f99d, e5c4c79cc4, 5c7b7c7d60). An optimisation of CompareAndSwap/Swap (06e6b8efa4) was reverted (c112c0af13) after issue 70970 (regression test added 3f002abb60). unique.Make uses TypeFor (755c18ecdf).
Timeline.
- Go 1.24 — Done: all above. Left open: reverted optimisation of CompareAndSwap/Swap.
- Go 1.26 — Done: warning about incorrect unsafe usage documented (
1718828c81). Left open: nothing noted.
iter.Pull and range-over-func iterator types
Span: Go 1.22 → Go 1.28 · Status at HEAD: stable; documentation churn
- proposal #61897 iter: new package for iterators referenced by 2 of this thread's commits
- discussion user-defined iteration using range over func values listed in Go: A Documentary
- background Coroutines for Go listed in Go: A Documentary
Design. iter.Pull/Pull2 convert push iterators into pull iterators using coroutines (coroswitch) between goroutines. Hardening: next/yield called out of sequence would switch to the wrong goroutine with garbage values or crash the runtime; one bool now enforces strict alternation (ff743ce862, panics instead). Panics (1471978bac) and runtime.Goexit (9d2aeae72d) from the underlying iterator now propagate to the caller of next. Memory footprint reduced by grouping captured state in one struct (ba7b8ca336, one heap allocation instead of several; trade-off: unrelated state retained until stop).
Timeline.
- Go 1.22 — Done: race instrumentation fix for Pull2 (
f2d243db8f). Left open: behind GOEXPERIMENT=rangefunc. - Go 1.23 — Done: user-visible types
Seq/Seq2exposed without experiment (5a181c5042), double next/yield rejection (ff743ce862), panic propagation (1471978bac), Goexit propagation (9d2aeae72d), stop-before-next (1634fde4f9), doc from proposal (fe36ce669c). Left open: nothing noted. - Go 1.25 — Done: smaller Pull footprint (
ba7b8ca336). Left open: nothing noted. - Go 1.26/1.28 — Done: doc clarifications for yield after loop exit panics (
61dc7fe30d), examples (4794b781c1). Left open: nothing noted.
Classic map maintenance and maps package helpers (pre-swiss)
Span: Go 1.18 → Go 1.24 · Status at HEAD: old implementation deleted in 1.26
Earlier history: Hash map core: from the first hashmap.c to the 8-entry bucket design (Go 1.0 → Go 1.17) · Map growth, iteration order, overflow buckets and GC interaction (Go 1.0 → Go 1.12) · Map fast paths and compiler/runtime specialization (Go 1.1 → Go 1.17)
Design. Fixes and tweaks to runtime/map.go and compiler lowering before and during the swiss transition: map constants centralised in internal/abi (“makes it much easier to change the size of map buckets”, e22bd2348c), a regression where the load factor became 6 instead of 6.5 (25e48765a4), clear marks bucket slots empty so in-flight iterators terminate (231f290e51; second iterator-after-clear fix 1f8fa4941f), maps.Clone/Keys/Values speed ups in runtime (b5a7f2eef7, f283cba396, c6fd0c22dc), maps.Clone mid-grow bug (b303fb4855), aliasing (16d3040a84), bucket sizing (1b4cf43e42), empty map with interface key (d32b4798f8), real type size for key/value iteration (cc47df017d), “user” vs “system” throws so concurrent map misuse is fatal (29bbca5c2c).
Timeline.
- Go 1.19 — Done:
29bbca5c2c. Left open: nothing noted. - Go 1.21 — Done:
e22bd2348c,231f290e51, maps helpers. Left open: nothing noted. - Go 1.22 — Done:
25e48765a4,b303fb4855,16d3040a84,d32b4798f8,3bc28402fa(built-inclear). Left open: nothing noted. - Go 1.24 — Done:
1b4cf43e42,1f8fa4941f, linkname allowlists for map functions (8391579ece). Left open: swiss replacement.
Changes in this slice that are not part of a thread are listed on the release pages: 1.18 · 1.19 · 1.20 · 1.22 · 1.23 · 1.24 · 1.27