Pocket Gophers

Is handling dynamic JSON in Go really this difficult?

Say you have a JSON array with heterogenous types like:

[
	["Gopher Plush", 5],
	["Gopher Sticker", 77]
]

that you want to unmarshal into a slice of:

Item struct {
	Name     string
	Quantity int
}

This is just one example of dynamic JSON where unmarshaling into a tagged struct just doesn’t work. You may also have to deal with JSON that represents the same thing as an array of array and sometimes as an array of objects or maybe objects that tell you what type they are as a field of that same object. Worst of all, you feel like an idiot struggling to do something that would take 10s in python or ruby.

Do you have to descend into interface{} hell?

… and end up with Go that looks like:

func (item *Item) UnmarshalJSON(b []byte) error {
	var inner []interface{}
	err := json.Unmarshal(b, &inner)
	if err != nil {
		return err
	}

	if got, expected := len(inner), 2; got != expected {
		return fmt.Errorf("expected length %d, got %d",
			expected, got)
	}

	// fill in each field of the item
	var ok bool

	item.Name, ok = inner[0].(string)
	if !ok {
		return fmt.Errorf("Name is a %T: %v",
			inner[0], inner[0])
	}

	quantity, ok := inner[1].(float64)
	if !ok {
		return errors.New("Quantity is not a number")
	}
	if float64(int(quantity)) != quantity {
		return errors.New("Quantity is not an int")
	}
	item.Quantity = int(quantity)

	return nil
}

While this code works, it is fraught with difficulties:

What if you could confidently handle any JSON in an efficient and idiomatic way?

What if you could turn that overly long, unmaintainable code into:

func (item *Item) UnmarshalJSON(b []byte) error {
	inner := []interface{}{&item.Name, &item.Quantity}

	err := json.Unmarshal(b, &inner)
	if err != nil {
		return err
	}

	expected := len(inner)
	got := len(inner)
	if got != expected {
		return fmt.Errorf("expected length %d, got %d",
			expected, got)
	}

	return nil
}

Not only is this code shorter, but now:

The truth is, the more dynamic the JSON is, the more Go you have to write to deal with the variations. What if your solution was perfectly balanced with how dynamic the JSON it handles is?

Learn how to confidently handle any JSON with the Pocket Gophers’ Guide to JSON.

You’ll learn a universal approach to deal with any JSON that leverages encoding/json and other JSON tools in the Go ecosystem along with how to evaluate your implementation.

The Pocket Gophers’ Guide to JSON includes a PDF that explains the approach, the tools, and the evaluation method. The guide also includes examples based on real-world JSON with many solutions (where possible) with detailed evaluations.

You’ll learn how to efficiently handle your JSON with confidence that your solution balances the implementation with how dynamic the JSON is and at the same time be idiomatic Go.

Just fire up your editor and buy The Pocket Gophers’ Guide to JSON. Then you’ll be confidently handling any JSON today.

Early Access Special

Buy Now For $50 $31

Early Access?

This guide is not yet complete. I intend every version of this guide to be useful. It is already useful and will only become more so. I reduced the price to reflect this and the additional effort required to keep up as the guide is updated.

New additions will be listed in the changelog include with the guide. Revisions to existing content will be listed if the changes make the content worth revisiting.

Though I have worked hard to make sure the text and examples work and are easy to understand, it’s still possible that you will find code or text that does not work for you or that you don’t understand. When this happens, please email me at nathan@pocketgophers.com and I will try to fix the problem. Questions and example suggestions are also welcome.

The Examples

The examples included in this guide are based on real-world JSON. The examples are designed to let you quickly explore the alternatives, and even implement your own.

Each example has tests, benchmarks, and an explanation of what each solution intends to accomplish. This early-access release has a subset of the examples and solutions that will be included in the full version. So far, the examples are:

string
Convert between a JSON string and a Go string. Has three solutions.
array-struct
Convert between a heterogeneous JSON array and a Go struct. Has four solutions.
object-or-array
The JSON value could be an array of arrays or an array of objects. In both cases they represent the same thing. Has four solutions.
string-or-int
The JSON value could be a string or an integer, both representing the same thing. Has three solutions.
filter-array
Filter and sort an array of heterogeneous objects, returning the complete objects (with unknown fields) that pass the filter. Has four solutions.
merge-objects
Merge components array of all objects with the same id. Has three solutions.

Early Access Special

Buy Now For $50 $31

UPDATE 2018-02-07: The next release, v0.3, will complete what I think of as the core of the guide — a comprehensive view of encoding/json. After that I just need to make sure all the tricks found in JSON References are covered and integrate when and how other JSON Packages should be used (github.com/a8m/djson is already included). I expect to release it in the next couple of weeks.

Questions?

Can I get a sample?
Yes. The sample contains a subset of the released version.
I'm having problems with some JSON. Will you help me?
Maybe. Email me the details and I’ll see what I can do.
How can I find out when the full version is released?
Subscribe to keep up with all Pocket Gophers' content, including when the full version of this guide is released.
How can I contact you?
Send me an email: nathan@pocketgophers.com