eli.thegreenplace.net

Go internals: capturing loop variables in closures

The Go wiki has a page titled CommonMistakes. Amusingly, it only lists a single entry at this time - using goroutines on loop iterator variables, providing this example: for _, val := range values { go func() { fmt.Println(val) }() } This will print the last value in values, len(values) times. The fix is very simple: // assume the type of each value is string for _, val := range values { go func(val string) { fmt.Println(val) }(val) } Being aware of the fix is sufficient to be able to write correct Go programs. However, if...

eli.thegreenplace.net

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) is very useful in cryptography and other domains. According to Wikipedia, its origin and name come from this riddle in a 3rd century book by a Chinese mathematician: There are certain things whose number is unknown. If we count them by threes, we have two left over; by fives, we have three left over; and by sevens, two are left over. How many things are there? Mathematically, this is a system of linear congruences. In this post we'll go through a simple proof of the existence of a solution. It also...

eli.thegreenplace.net

Passing callbacks and pointers to Cgo

Cgo enables Go programs to invoke C libraries or any other library that exposes a C API. As such, it's a important part of a Go programmer's toolbox. Using Cgo can be tricky, however, especially when passing pointers and callback functions between Go and C code. This post discusses an end-to-end example that covers: Basic usage of Cgo, including linking a custom C library into the Go binary. Passing structs from Go to C. Passing Go functions to C and arranging C to call them back later. Safely passing arbitrary Go data to C code, which can...

eli.thegreenplace.net

Summary of reading: April - June 2019

"In Praise of Slowness" by Carl Honoré - talks about the benefits of slowing down in different aspects of life. The book has some good ideas in it, but it's wildly drawn out, mixing in a lot of irrelevant information in order to reach a "book-worthy" page count. "SQL Queries for Mere Mortals" by John L. Viescas and Michael J. Hernandez - a very slow and gentle introduction to SQL. A good book for SQL newbies. My main criticism is the somewhat low signal-to-noise ratio; at 750 pages this book really doesn't cover...

eli.thegreenplace.net

SQL inner and outer joins

If you store data in a relational database, it's good practice to have the data normalized. This typically requires splitting data to multiple tables that are logically connected through keys. As a result, most non-trivial queries require joins on multiple tables to gather all the interesting columns. This post is a brief tour of SQL joins, focusing on the differences between inner and outer joins. Cross join To understand SQL joins, it's best to start with cross joins, because they are the simplest combination of tables supported by SQL. A...

eli.thegreenplace.net

Summary of reading: January - March 2019

"Cuckoo's Egg" by Clifford Stoll - a detailed account of the author's following a hacker breaking into pre-internet computer networks (in the 1980s). Very interesting historical perspective on computing and early security concerns - how simple and naive those times were! I wish the book would be shorter though. "Educated: A Memoir" by Tara Westover - the author grew up in a survivalist Mormon family in Idaho, with zero education until late teens, and parents who refused any formal contact with the establishment (no birth...

eli.thegreenplace.net

Does a concrete type implement an interface in Go?

A very common question that comes up in Go forums is "how to I check that some type implements a certain interface?". A common immediate reaction is that the question makes no sense, because Go is statically typed, so the compiler already knows whether a type implements an interface or not. But it turns out that the question in the general sense is more nuanced, and it's worth spending some time understanding the variations folks are usually interested in. Let's start with a basic example. Assume this Munger interface that has...

eli.thegreenplace.net

GitHub webhook payload as a cloud function

Back in 2014, I wrote a post describing a simple payload server for GitHub webhooks, using Python 3. That server could be deployed to any VPS listening on a custom port. Now it's 2019, and deploying servers to VPSs doesn't make me feel hip enough. All the cool kids are into serverless now, so I decided to rewrite the same payload server in Go and deploy it as a Google Cloud Function. This brief post can serve as a basic tutorial on how to do it. I assume you already have a GCP account (there's a free tier), and the gcloud command-line tool is...

eli.thegreenplace.net

Go JSON Cookbook

Recently I've gotten into answering Go questions on StackOverflow, and one of the patterns I noticed are many repetitive questions about JSON processing. The goal of this post is to collect a "cookbook" of JSON processing code and examples; think of it as a vastly expanded version of the JSON gobyexample page. It's a living document - I will update it once in a while when I find new patterns/problems people ask about. The code samples here should be reasonably self-contained; if you want actual code, full go run-able files are...

eli.thegreenplace.net

On concurrency in Go HTTP servers

Go's built-in net/http package is convenient, solid and performant, making it easy to write production-grade web servers. To be performant, net/http automatically employs concurrency; while this is great for high loads, it can also lead to some gotchas. In this post I want to explore this topic a bit. Ensuring safe concurrent access from handlers to data Let's start with a very simple example of a HTTP server that implements a table of counters accessible to the user. We can create counters (or set values of existing counters) with the...

eli.thegreenplace.net

Summary of reading: October - December 2018

"Wonder" by R.J. Palacio - school-age story about a boy with severe facial deformities who started going to 5th grade after being home-schooled earlier in life. A bit of bullying, lots of kindness, and interesting insights into the minds of 10-year-olds. "The Story of Human Language" by John McWhorter (audio course) - comprehensive introduction to linguistics and human languages. I've read Prof. McWhorter's books before (e.g. "Power of Babel"), but listening to it is a different experience. There's a lot of...

eli.thegreenplace.net

Beware of copying mutexes in Go

Suppose we have a struct that contains a map, and we want to modify the map in a method. Here's a simple example: package main import "fmt" type Container struct { counters map[string]int } func (c Container) inc(name string) { c.counters[name]++ } func main() { c := Container{counters: map[string]int{"a": 0, "b": 0}} doIncrement := func(name string, n int) { for i := 0; i < n; i++ { c.inc(name) } } doIncrement("a", 100000) fmt.Println(c.counters) } A Container...