(Generic) Functional Options Pattern

Targeted Go version: 1.18

Author(s): Changkun Ou

Permalink: https://golang.design/research/generic-option

The widely used self-referential function pattern as options, originally proposed by Rob Pike¹, allows us to design a flexible set of APIs to help arbitrary configurations and initialization of a struct. However, when such a pattern is cumbersome when we use one option to support multiple types. This article investigates how the latest Go generics design could empower a refreshed “generic” functional options pattern and show what improvements in the future version of Go could better support such a pattern.

The Functional Options Pattern

In the dotGo 2014, Dave Cheney² well explained the motivation and the use of self-referential functional options pattern in addition to the original thoughts from Rob Pike. Let's recall the key idea briefly.

Assume we have a struct A and it internally holds two user-customizable fields v1, v2:

1type A struct {
2    v1 int
3    v2 int
4}

Typically, we could make v1 and v2 to be public fields, and let users of this struct edit them directly, but this may create difficult compatibility issues to deprecate a field without breaking anything. Another side effect of having public fields is we cannot guarantee the concurrent safty from the user level: there is no way to prevent people from directly editing the public fields.

Instead, we could define a type Option to a self referential function func(*A):

1type Option func(*A)

Then, in order to change the private fields v1 and v2, two functions V1 and V2 that returns an Option can be written as follows:

 1func V1(v1 int) Option {
 2    return func(a *A) {
 3        a.v1 = v1
 4    }
 5}
 6
 7func V2(v2 int) Option {
 8    return func(a *A) {
 9        a.v2 = v2
10    }
11}

With these functions, the initial settings of an A object could be created by a NewA function that consumes arbitrary number of options:

1func NewA(opts ...Option) *A {
2    a := &A{}
3    for _, opt := range opts {
4        opt(a)
5    }
6    return a
7}

For example, the following four different usages both work:

1fmt.Printf("%#v\n", NewA())               // &A{v1:0, v2:0}
2fmt.Printf("%#v\n", NewA(V1(42)))         // &A{v1:42, v2:0}
3fmt.Printf("%#v\n", NewA(V2(42)))         // &A{v1:0, v2:0}
4fmt.Printf("%#v\n", NewA(V1(42), V2(42))) // &A{v1:42, v2:0}

This is also super easy to deprecate an option, because we can simply let an existing option function not effecting anymore. For instance:

 1type A struct {
 2    v1 int
 3-    v2 int
 4+    // Removed, now moved to v3.
 5+    // v2 int
 6    v3 int
 7}
 8
 9type Option func(*A)
10
11func V1(v1 int) Option {
12    return func(a *A) {
13        a.v1 = v1
14    }
15}
16
17+// Deprecated: Use V3 instead.
18func V2(v2 int) Option {
19    return func(a *A) {
20-        a.v2 = v2
21+        // no effects anymore
22+        // a.v2 = v2
23    }
24}
25
26+func V3(v3 int) Option {
27+    return func(a *A) {
28+        a.v3 = v3
29+    }
30+}

The previous code that uses V2 will have a smooth transition without any breaks.

The Problem at Scale

Such a functional option pattern scales very ugly when we have tons of options and multiple types in the same package that need customization.

Let's explain in more depth with another example. When types A and B sharing similar fields and both need options to customize:

1type A struct {
2    v1 int
3}
4
5type B struct {
6    v1 int
7    v2 int
8}

We will have to define two types of options separately for A and B. There is no easy way to write a unified functional option that both works for A and B, and for the same field v1, we need two versions of options V1ForA and V1ForB to manipulate:

 1type OptionA func(a *A)
 2type OptionB func(a *B)
 3
 4func V1ForA(v1 int) OptionA {
 5    return func(a *A) {
 6        a.v1 = v1
 7    }
 8}
 9
10func V1ForB(v1 int) OptionB {
11    return func(b *B) {
12        b.v1 = v1
13    }
14}
15
16func V2ForB(v2 int) OptionB {
17    return func(b *B) {
18        b.v2 = v2
19    }
20}
21
22func NewA(opts ...OptionA) *A {
23    a := &A{}
24
25    for _, opt := range opts {
26        opt(a)
27    }
28    return a
29}
30
31func NewB(opts ...OptionB) *B {
32    b := &B{}
33
34    for _, opt := range opts {
35        opt(b)
36    }
37    return b
38}

In this way, whenever we need create a new A or B, we could:

1fmt.Printf("%#v\n", NewA())                       // &A{v1:0}
2fmt.Printf("%#v\n", NewA(V1ForA(42)))             // &A{v1:42}
3fmt.Printf("%#v\n", NewB())                       // &B{v1:0, v2:0}
4fmt.Printf("%#v\n", NewB(V1ForB(42)))             // &B{v1:42, v2:0}
5fmt.Printf("%#v\n", NewB(V2ForB(42)))             // &B{v1:0, v2:42}
6fmt.Printf("%#v\n", NewB(V1ForB(42), V2ForB(42))) // &B{v1:42, v2:42}

Although the above workaround is possible, but the actual naming and usage really feels cambersum, especially when these options are in a separate package where we have to supply the package name when dot import is not used (assume the package name is called pkgname):

1fmt.Println(pkgname.NewA())
2fmt.Println(pkgname.NewA(pkgname.V1ForA(42)))
3fmt.Println(pkgname.NewB())
4fmt.Println(pkgname.NewB(pkgname.V1ForB(42)))
5fmt.Println(pkgname.NewB(pkgname.V2ForB(42)))
6fmt.Println(pkgname.NewB(pkgname.V1ForB(42), pkgname.V2ForB(42)))

Can we do something better?

Using Interfaces

A quick solution to deal with this is to use an interface where an interface that commonly represents A and B:

 1type A struct {
 2	v1 int
 3}
 4
 5type B struct {
 6	v1 int
 7	v2 int
 8}
 9
10type Common interface {
11	/* ... */
12}

Then we can write options as follows using a Common interface, and type switches:

 1type Option func(c Common)
 2
 3func V1(v1 int) Option {
 4	return func(c Common) {
 5		switch x := c.(type) {
 6		case *A:
 7			x.v1 = v1
 8		case *B:
 9			x.v1 = v1
10		default:
11			panic("unexpected use")
12		}
13	}
14}
15
16func V2(v2 int) Option {
17	return func(c Common) {
18		switch x := c.(type) {
19		case *B:
20			x.v2 = v2
21		default:
22			panic("unexpected use")
23		}
24	}
25}
26
27func NewA(opts ...Option) *A {
28	a := &A{}
29
30	for _, opt := range opts {
31		opt(a)
32	}
33	return a
34}
35
36func NewB(opts ...Option) *B {
37	b := &B{}
38
39	for _, opt := range opts {
40		opt(b)
41	}
42	return b
43}

Without further changes, one can use V1 both for A and B, which is a quite simplification from the previous use already:

1fmt.Printf("%#v\n", NewA())               // &A{v1:0}
2fmt.Printf("%#v\n", NewA(V1(42)))         // &A{v1:42}
3fmt.Printf("%#v\n", NewB())               // &B{v1:0, v2:0}
4fmt.Printf("%#v\n", NewB(V1(42)))         // &B{v1:42, v2:0}
5fmt.Printf("%#v\n", NewB(V2(42)))         // &B{v1:0, v2:42}
6fmt.Printf("%#v\n", NewB(V1(42), V2(42))) // &B{v1:42, v2:42}

However, not everything goes as expected. There is a heavy cost for this type of functional options pattern: safety.

Let's imagine when we accidentally use V2 in NewA, what will happen?

1fmt.Println(NewA(V2(42)))

panic: unexpected use

goroutine 1 [running]:
main.main.func6({0x104f38a20?, 0x14000122110?})

Clearly, code like this will result in a panic at runtime, because there is no safety mechanism to prevent not using V2 in NewA. Furthermore, from the caller's perspective, unless we further look into the implementation of V2, there is no way we could tell whether we can use V2 in NewA or not.

Using Generics (and Make Call Safer)

With the Go 1.18's generics, we could consider using a generic version of options to simplify the previously mentioned available options further and guarantee the safety of calls.

Let's now consider the same types A and B:

1type A struct {
2    v1 int
3}
4
5type B struct {
6    v1 int
7    v2 int
8}

Then, instead of defining a direct functional option or using a common interface, we define a generic option Option[T] that accepts A or B as its type parameters. In this case, the self-referred function is also a parameterized function func(*T):

1type Option[T A | B] func(*T)

We can carefully constrain the type parameters of the option functions V1 and V2. Specifically, In the option function V1, is designed to use for either type A or B, therefore constraining its type parameter T also limits the possible return types of V1 to be either Option[A] or Option[B]; in the option function V2, we only intended to let it is used in type B. Hence we could permit B as its type parameter, and therefore the compiler will only instantiate the version of V2 that returns Option[B].

 1func V1[T A | B](v1 int) Option[T] {
 2	return func(a *T) {
 3		switch x := any(a).(type) {
 4		case *A:
 5			x.v1 = v1
 6		case *B:
 7			x.v1 = v1
 8		default:
 9			panic("unexpected use")
10		}
11	}
12}
13
14func V2[T B](v2 int) Option[T] {
15	return func(a *T) {
16		switch x := any(a).(type) {
17		case *B:
18			x.v2 = v2
19		default:
20			panic("unexpected use")
21		}
22	}
23}

Furthermore, in the constructor of A and B. We only permit their dedicated options, such as NewA only permits type A and NewB only allow type B as their type parameters:

 1func NewA[T A](opts ...Option[T]) *T {
 2	t := new(T)
 3	for _, opt := range opts {
 4		opt(t)
 5	}
 6	return t
 7}
 8
 9func NewB[T B](opts ...Option[T]) *T {
10	t := new(T)
11	for _, opt := range opts {
12		opt(t)
13	}
14	return t
15}

On the call side, we have:

1fmt.Printf("%#v\n", NewA())                     // &main.A{v1:0}
2fmt.Printf("%#v\n", NewA(V1[A](42)))            // &main.A{v1:42}
3fmt.Printf("%#v\n", NewB())                     // &main.B{v1:0, v2:0}
4fmt.Printf("%#v\n", NewB(V1[B](42)))            // &main.B{v1:42, v2:0}
5fmt.Printf("%#v\n", NewB(V2[B](42)))            // &main.B{v1:0, v2:42}
6fmt.Printf("%#v\n", NewB(V1[B](42), V2[B](42))) // &main.B{v1:42, v2:42}

With this design, the user of these APIs is safe because it is guaranteed by the compiler at compile-time, to disallow its misuse by the following errors:

 1// ERROR: B does not implement A
 2_ = NewA(V2[B](42))
 3// ERROR: A does not implement B
 4_ = NewA(V2[A](42))
 5// ERROR: type Option[B] of V2[B](42) does not match
 6// inferred type Option[A] for Option[T]
 7_ = NewB(V1[A](42), V2[B](42))
 8// ERROR: type Option[A] of V2[A](42) does not match
 9// inferred type Option[B] for Option[T]
10_ = NewB(V1[B](42), V2[A](42))

Conclusion

This article discussed how generics could empower a future version of functional option pattern to make such a pattern more compact and safer to use. However, there is one thing left that we could not optimize yet, which is the compiler type inference for the readability and simplicity.

In the last generics functional option design, we have calls similar to:

1NewA(V1[A](42)))
2NewB(V1[B](42), V2[B](42))

This could become a little bit stutter when these functions and options are from a different package, say pkgname. In this case, we will have to write:

1pkgname.NewA(pkgname.V1[pkgname.A](42)))

One may wonder: can't we avoid writing the type parameters of V1 and V2?

Indeed, there is only one possibility for V1 to satisfy the NewA's type constraints because NewA only accepts type A as type parameters. If V1 is used as the argument of NewA, then V1 must return Option[A], and therefore the type parameter of V1 must be A; similar to V2.

With this observation, we could simplify our code from:

1pkgname.NewA(pkgname.V1[pkgname.A](42))
2pkgname.NewB(pkgname.V1[pkgname.B](42), pkgname.V2[pkgname.B](42))

to

1pkgname.NewA(pkgname.V1(42))
2pkgname.NewB(pkgname.V1(42), pkgname.V2(42))

With this simplification, on the caller side, we see a sort of magic function V1 as an option, which can be used both for NewA and NewB. Unfortunately, with the current Go 1.18 generics implementation, this type of inference is not yet supported.

We have created an issue³ for the Go team and see if this type of optimization could be possible without introducing any other flaws. Let's looking forward to it!

References