eli.thegreenplace.net

Go internals: capturing loop variables in closures

The Go wiki has a page titled CommonMistakes. Amusingly, it only lists a single entry at this time - using goroutines on loop iterator variables, providing this example: for _, val := range values { go func() { fmt.Println(val) }() } This will print the last value in values, len(values) times. The fix is very simple: // assume the type of each value is string for _, val := range values { go func(val string) { fmt.Println(val) }(val) } Being aware of the fix is sufficient to be able to write correct Go programs. However, if...

eli.thegreenplace.net

The Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) is very useful in cryptography and other domains. According to Wikipedia, its origin and name come from this riddle in a 3rd century book by a Chinese mathematician: There are certain things whose number is unknown. If we count them by threes, we have two left over; by fives, we have three left over; and by sevens, two are left over. How many things are there? Mathematically, this is a system of linear congruences. In this post we'll go through a simple proof of the existence of a solution. It also...

eli.thegreenplace.net

Passing callbacks and pointers to Cgo

Cgo enables Go programs to invoke C libraries or any other library that exposes a C API. As such, it's a important part of a Go programmer's toolbox. Using Cgo can be tricky, however, especially when passing pointers and callback functions between Go and C code. This post discusses an end-to-end example that covers: Basic usage of Cgo, including linking a custom C library into the Go binary. Passing structs from Go to C. Passing Go functions to C and arranging C to call them back later. Safely passing arbitrary Go data to C code, which can...

eli.thegreenplace.net

Summary of reading: April - June 2019

"In Praise of Slowness" by Carl Honoré - talks about the benefits of slowing down in different aspects of life. The book has some good ideas in it, but it's wildly drawn out, mixing in a lot of irrelevant information in order to reach a "book-worthy" page count. "SQL Queries for Mere Mortals" by John L. Viescas and Michael J. Hernandez - a very slow and gentle introduction to SQL. A good book for SQL newbies. My main criticism is the somewhat low signal-to-noise ratio; at 750 pages this book really doesn't cover...

eli.thegreenplace.net

SQL inner and outer joins

If you store data in a relational database, it's good practice to have the data normalized. This typically requires splitting data to multiple tables that are logically connected through keys. As a result, most non-trivial queries require joins on multiple tables to gather all the interesting columns. This post is a brief tour of SQL joins, focusing on the differences between inner and outer joins. Cross join To understand SQL joins, it's best to start with cross joins, because they are the simplest combination of tables supported by SQL. A...

eli.thegreenplace.net

Summary of reading: January - March 2019

"Cuckoo's Egg" by Clifford Stoll - a detailed account of the author's following a hacker breaking into pre-internet computer networks (in the 1980s). Very interesting historical perspective on computing and early security concerns - how simple and naive those times were! I wish the book would be shorter though. "Educated: A Memoir" by Tara Westover - the author grew up in a survivalist Mormon family in Idaho, with zero education until late teens, and parents who refused any formal contact with the establishment (no birth...

eli.thegreenplace.net

Does a concrete type implement an interface in Go?

A very common question that comes up in Go forums is "how to I check that some type implements a certain interface?". A common immediate reaction is that the question makes no sense, because Go is statically typed, so the compiler already knows whether a type implements an interface or not. But it turns out that the question in the general sense is more nuanced, and it's worth spending some time understanding the variations folks are usually interested in. Let's start with a basic example. Assume this Munger interface that has...

eli.thegreenplace.net

GitHub webhook payload as a cloud function

Back in 2014, I wrote a post describing a simple payload server for GitHub webhooks, using Python 3. That server could be deployed to any VPS listening on a custom port. Now it's 2019, and deploying servers to VPSs doesn't make me feel hip enough. All the cool kids are into serverless now, so I decided to rewrite the same payload server in Go and deploy it as a Google Cloud Function. This brief post can serve as a basic tutorial on how to do it. I assume you already have a GCP account (there's a free tier), and the gcloud command-line tool is...

eli.thegreenplace.net

Go JSON Cookbook

Recently I've gotten into answering Go questions on StackOverflow, and one of the patterns I noticed are many repetitive questions about JSON processing. The goal of this post is to collect a "cookbook" of JSON processing code and examples; think of it as a vastly expanded version of the JSON gobyexample page. It's a living document - I will update it once in a while when I find new patterns/problems people ask about. The code samples here should be reasonably self-contained; if you want actual code, full go run-able files are...

eli.thegreenplace.net

On concurrency in Go HTTP servers

Go's built-in net/http package is convenient, solid and performant, making it easy to write production-grade web servers. To be performant, net/http automatically employs concurrency; while this is great for high loads, it can also lead to some gotchas. In this post I want to explore this topic a bit. Ensuring safe concurrent access from handlers to data Let's start with a very simple example of a HTTP server that implements a table of counters accessible to the user. We can create counters (or set values of existing counters) with the...

eli.thegreenplace.net

Summary of reading: October - December 2018

"Wonder" by R.J. Palacio - school-age story about a boy with severe facial deformities who started going to 5th grade after being home-schooled earlier in life. A bit of bullying, lots of kindness, and interesting insights into the minds of 10-year-olds. "The Story of Human Language" by John McWhorter (audio course) - comprehensive introduction to linguistics and human languages. I've read Prof. McWhorter's books before (e.g. "Power of Babel"), but listening to it is a different experience. There's a lot of...

eli.thegreenplace.net

Beware of copying mutexes in Go

Suppose we have a struct that contains a map, and we want to modify the map in a method. Here's a simple example: package main import "fmt" type Container struct { counters map[string]int } func (c Container) inc(name string) { c.counters[name]++ } func main() { c := Container{counters: map[string]int{"a": 0, "b": 0}} doIncrement := func(name string, n int) { for i := 0; i < n; i++ { c.inc(name) } } doIncrement("a", 100000) fmt.Println(c.counters) } A Container...

eli.thegreenplace.net

Type erasure and reification

In this post I'd like to discuss the concepts of type erasure and reification in programming languages. I don't intend to dive very deeply into the specific rules of any particular language; rather, the post is going to present several simple examples in multiple languages, hoping to provide enough intuition and background for a more serious study, if necessary. As you'll see, the actual concepts are very simple and familiar. Deeper details of specific languages pertain more to the idiosyncrasies of those languages' semantics and...

eli.thegreenplace.net

Type inference

Type inference is a major feature of several programming languages, most notably languages from the ML family like Haskell. In this post I want to provide a brief overview of type inference, along with a simple Python implementation for a toy ML-like language. Uni-directional type inference While static typing is very useful, one of its potential downsides is verbosity. The programmer has to annotate values with types throughout the code, which results in more effort and clutter. What's really annoying, though, is that in many cases these...

eli.thegreenplace.net

Unification

In logic and computer science, unification is a process of automatically solving equations between symbolic terms. Unification has several interesting applications, notably in logic programming and type inference. In this post I want to present the basic unification algorithm with a complete implementation. Let's start with some terminology. We'll be using terms built from constants, variables and function applications: A lowercase letter represents a constant (could be any kind of constant, like an integer or a string) An uppercase letter...