Bypassing Go visibility rules (with generics!)

Posted on Feb 1, 2022

(You can paste all snippets into the Go playground.)

Bypassing the visibility rules has always been easy for structs, and there are several approaches that I will explain in this post.

The first approach involves finding the offset of the field in the struct using reflection, then the unsafe package to create an arbitrary pointer to the field.

-- go.mod --
module tmp

go 1.18


-- unsafe/some_package/some_package.go --
package some_package

type S struct {
	Number int32
	Letter rune
	secret string
}

func NewS() S {
	return S{secret: "secret string"}
}


-- main.go --
package main

import (
	"fmt"
	"reflect"
	"unsafe"

	"tmp/unsafe/some_package"
)

func main() {
	s := some_package.NewS()

	// Find the "secret" field.
	f, ok := reflect.TypeOf(s).FieldByName("secret")
	if !ok {
		panic("could not find field")
	}

	// Get the address of the struct.
	ptr := unsafe.Pointer(&s)

	// Add the offset of the field within the struct to
	// the address of the struct, effectively getting the
	// address of the field.
	ptr = unsafe.Add(ptr, f.Offset)

	// ptr now points to the address of "secret" in the
	// struct, so it can be cast to *string and then
	// dereferenced.
	strp := (*string)(ptr)
	fmt.Println(*strp)

	// Or just:
	fmt.Println(*(*string)(unsafe.Add(unsafe.Pointer(&s), f.Offset)))

	// Or just use reflect.Value, which is basically the same:
	fmt.Println(*(*string)(unsafe.Pointer(
		reflect.
			ValueOf(&s).
			Elem().
			FieldByName("secret").
			Addr().
			Pointer())))
}

This approach is fairly safe as the reflect package should be able to provide the exact offset of the field, and usage of the unsafe package is extremely easy to audit and reason about. It’s just wrong, you know. Like, philosophically.

Another approach is to write an identical struct and use the unsafe package to cast a pointer to the struct that contains the unexported field to a pointer of this helper struct:

-- go.mod --
module tmp

go 1.18


-- unsafe/some_package/some_package.go --
package some_package

type S struct {
	Number int32
	Letter rune
	secret string
}

func NewS() S {
	return S{secret: "secret string"}
}


-- main.go --
package main

import (
	"fmt"
	"unsafe"

	"tmp/unsafe/some_package"
)

func main() {
	s := some_package.NewS()

	// This struct should have the same memory layout as
	// some_package.S. This is, however, not guaranteed in any
	// way by the Go spec.
	// Still, as far as I'm aware the Go compiler does not
	// reorder struct fields and adds adequate padding.
	// If it did reorder the fields, it would be very likely that
	// they would be reordered in the same way, anyway, so this
	// is probably fine. But it's very fucky.
	type identical struct {
		_      int32
		_      rune
		secret string
	}

	// Now it is possible to take the address of s,
	ptr := unsafe.Pointer(&s)

	// Cast this pointer to *identical,
	i := (*identical)(ptr)

	// And access the string field, which should be in the same
	// spot.
	fmt.Println(i.secret)

	// Or just:
	fmt.Println((*struct {
		_ int32
		_ rune
		s string
	})(unsafe.Pointer(&s)).s)
}

There are some limitations to this. For example, extracting this to a helper would pose a challenge, as such a function would need to accept interface{} as the type for the struct argument, so it would not be able to take a pointer to the parameter. Also, returning a pointer to the field or the value of the field would also be challenging.

The following two approaches do work, but they are not particularly ergonomic.

-- go.mod --
module tmp

go 1.18


-- unsafe/some_package/some_package.go --
package some_package

type S struct {
	Number int32
	Letter rune
	secret string
}

func NewS() S {
	return S{secret: "secret string"}
}


-- main.go --
package main

import (
	"fmt"
	"reflect"
	"tmp/unsafe/some_package"
	"unsafe"
)

// This approach takes the struct in two parameters, first the
// interface{} and then a pointer to its address, then uses the
// reflection approach from before to return a pointer to the
// field, which the caller can then cast to whatever type the
// field is.
// It's obviously annoying to call this function, as you need to
// pass the same parameter twice and then cast the output.
func field1(
	v interface{},
	ptr unsafe.Pointer,
	fname string,
) unsafe.Pointer {
	f, ok := reflect.TypeOf(v).FieldByName(fname)
	if !ok {
		panic(fmt.Sprintf("cannot find field %s in %T", fname, v))
	}
	return unsafe.Add(ptr, f.Offset)
}

// This approach takes a reflect.Value instead and uses that to
// take the address to the field.
// It's slightly better than the other approach but it still
// requires to cast an unsafe.Pointer.
func field2(v reflect.Value, fname string) unsafe.Pointer {
	f := v.Elem().FieldByName(fname)
	if f == (reflect.Value{}) {
		panic(fmt.Sprintf(
			"cannot find field %s in %s", fname, v.Type()))
	}
	return unsafe.Pointer(f.Addr().Pointer())
}

func main() {
	s := some_package.NewS()

	fmt.Println(*(*string)(field1(s, unsafe.Pointer(&s), "secret")))
	fmt.Println(*(*string)(field2(reflect.ValueOf(&s), "secret")))
}

Go 1.18 is introducing generics, which allows to write a function to get this done much more easily:

-- go.mod --
module tmp

go 1.18


-- unsafe/some_package/some_package.go --
package some_package

type S struct {
	Number int32
	Letter rune
	secret string
}

func NewS() S {
	return S{secret: "secret string"}
}


-- main.go --
package main

import (
	"fmt"
	"reflect"
	"tmp/unsafe/some_package"
	"unsafe"
)

// This is essentially the approach from the first example where
// the offset of the field is found using reflection and then a
// pointer is created pointing to the struct, to which the offset
// of the field is added.
// Here, however, instead of returning an unsafe.Pointer which
// the caller needs to cast, the cast can happen inside the
// function since it can return a generic type U.
func field[T any, U any](v *T, fname string) U {
	f, ok := reflect.TypeOf(v).Elem().FieldByName(fname)
	if !ok {
		panic(fmt.Sprintf("cannot find field %s in %T", fname, v))
	}
	return *(*U)(unsafe.Add(unsafe.Pointer(v), f.Offset))
}

func main() {
	s := some_package.NewS()

	// So accessing the field becomes as easy as:
	fmt.Println(field[some_package.S, string](&s, "secret"))

	// Unfortunately, I could not make the compiler infer all the
	// parameters. The following would not compile:
	//	var s string = field(&s, "secret")
	// Because the compiler cannot infer that U needs to be
	// string.
}

I wonder how long it will take for someone to write some package that has these kinds of unsafe and rule-bending functions. It would not add much new stuff to the ecosystem since these things can already happen, but it would certainly make it easier.

Also, please don’t write code that makes others wish some field was exported. Thanks.

🔎 Browse comments 💬 Post a new comment