Chapter 5 · Data Structures

5.2 Strings and Zero-Copy Conversion

Strings share their origin with slices: in the runtime both are “a header plus memory living elsewhere” (for the layout, see 5.1.1). But a string is read-only and immutable, which makes it its own category. Immutability buys us sharing and copy-free use, and it brings one direct cost: converting between string and []byte copies the bytes by default. This section opens up that copy, looks at the cases where the compiler can elide it, then turns to the manual zero-copy tools Go 1.20 provides and the safety contract that comes with them.

5.2.1 Converting Between Strings and []byte

String immutability brings one direct cost: converting between string and []byte copies the bytes by default. The reason is immutability itself. A []byte is writable, so if we let it point straight at the underlying bytes of some string, modifying the []byte would mean modifying that “immutable” string, breaking every sharing assumption. So the runtime dutifully memmoves a fresh copy:

1
2
3
4
5
6
7
// runtime: []byte → string (string.go, excerpt)
func slicebytetostring(buf *tmpBuf, ptr *byte, n int) string {
    // ...
    p := mallocgc(uintptr(n), nil, false) // allocate new memory
    memmove(p, unsafe.Pointer(ptr), uintptr(n)) // copy the bytes
    return unsafe.String((*byte)(p), n)
}

This copy can be noticeable on a hot path. Fortunately the compiler recognizes a few patterns where “the bytes are read immediately after conversion and cannot be modified” and elides the copy. The two most common are using a []byte as a temporary map key for a lookup, and ranging directly over []byte(s):

1
2
3
4
5
var m map[string]int
_ = m[string(b)]      // compiler: the temporary string is only used to look up the map, no copy needed (uses slicebytetostringtmp)
for i, c := range []byte(s) { // compiler: only iterated, not retained, no need to actually build a slice
    _, _ = i, c
}

To do a manual zero-copy conversion in your own code, Go 1.20 gives the official tools: unsafe.String(*byte, len) views a span of bytes as a string, unsafe.StringData(string) *byte takes a string’s underlying pointer, and unsafe.Slice / unsafe.SliceData are the slice-side counterparts. They replace the older, fragile practice of hand-assembling a header through reflect.StringHeader / reflect.SliceHeader (that practice is not reliable in the presence of GC movement or changes in field alignment). The price is that you must guarantee yourself that the bytes are no longer modified after the conversion, otherwise you have punctured the immutability contract:

1
2
// zero-copy, and only valid when you can guarantee b is read-only from here on
s := unsafe.String(unsafe.SliceData(b), len(b))