Demonstrating Go’s Easy C Interop

EDIT: Own an Android device? Check out Gallus on the Play Store. It’s like Instagram brought Hyperlapse to Android, only a lot better.

Some feedback from the last entry (on Go’s most powerful feature being the least sexy) made clear that there is a bit of confusion regarding how Go plays nicely with C. While there are some online docs for cgo, the examples are very limited, leaving many assuming that Go’s C integration is limited to calling some stdlib functions.

So I’m going to hash out a quick example of a hypothetical Go program that I’ve adorned with a bit of C.

For the purposes of brevity, I’m eschewing most error or bounds checking, or any manner of security. This is not production ready code, but simply shows how easily C and Go live side by side in the same app, and how simple it is to leverage this flexibility.

I’m using Ubuntu 12.0.4 LTS for this example, though this can be easily adapted to other Linux distributions, Windows, OSX, or any other platform where the dependencies can be satisfied (e.g. install gcc through cygwin on Windows).

You need to have Go 1.1+ installed. If you don’t, use GVM to grab and install it, or just download the latest binaries, copy them to /opt/go (set GOROOT to that directory, and add /opt/go/bin to your path), or just run the installer in Windows or OSX. You will also need git, build-essential (for the gcc toolchain used by cgo), and hg (e.g. sudo apt-get install git build-essential mercurial).

Once we have all of that, let’s configure our basic workspace for 0gc (note that’s zero-the-number-g-c), preparing the groupcache dependency that we’re going to leverage.

mkdir ~/0gc
export GOPATH=~/0gc
cd ~/0gc
go get github.com/golang/groupcache
mkdir ~/0gc/src/github.com/0gc

Now we’ll drop the following code in the file ~/0gc/src/github.com/0gc/main.go 

If you’ve followed along correctly (and I didn’t miss anything in my instructions), you should be able to run…

go run ~/0gc/src/github.com/0gc/main.go

…or alternately you can build and run via…

go build github.com/0gc
./0gc

…yielding an interactive console that uses the rather exciting Go groupcache (as discussed on HN, and as used extensively in Google’s back-end) to cache and orchestrate the complex reversing of strings. Ctrl-C or enter an empty key to exit.

Now we have our brilliant little go app, but we’d like to allow other applications on this and other machines to make use of the same cache source, so let’s add a simple zeromq request/response mechanism to our Go app.

If you don’t have the 0mq libraries and includes, do the following (obviously if you traveled here from future world, change to the appropriate version of all of the packages mentioned)-

cd ~ 
wget http://download.zeromq.org/zeromq-3.2.3.tar.gz
tar xvfz zeromq-3.2.3.tar.gz
cd zeromq-3.2.3
./configure
sudo make install

This installs the library to /usr/local/lib by default, so add that to your library path by adding, unsurprisingly, the line /usr/local/lib to the file /etc/ld.so.conf (e.g. …

sudo nano /etc/ld.so.conf

…add the line and then ctrl-O to save the file, ctrl-X to exit. Then execute…

sudo ldconfig

…so the change is incorporated).

While zeromq has some incomplete bindings for Go, for the purpose of this example let’s skip the middle man and add the following C code to the file ~/0gc/src/github/0gc/main.c (this code is a marginally altered version of 0mq’s example request/response example).

 

Since our project has become more complex, we need to stick to using the build command from here on out. However we’re not quite ready yet — we need to make a couple of changes to the original ~/0gc/src/github.com/0gc/main.go  file to tell Go that we’re using cgo, to hook in the C function and export the necessary Go export, and to add the link flag that tells Go to link in 0mq.

Now let’s build and run…

go build github.com/0gc
./0gc

…and now we will have a silent, concurrent 0mq req/response listener quietly serving up cache entries to remote callers alongside our interactive console. Note that we never had to explicitly detail the C files we use in our project, as cgo finds them wherever they hide.

Of particular interest here is line 4 which configures the linker (adding the library search path and linking zmq), and line 5 that imports the function that we’re going to use from C. Line 6 is the magic flag that tells cgo to play a part. Line 60 defines getValue as an exported function that can be called from C. Line 51 is where some magic happens, spinning off a goroutine that runs the C function we imported in line 5.

If you look back to main.c, on line 1 we included a file _cgo_export.h which is automatically generated by cgo on build, adding the basic Go rudiments (like GoString) along with the uniques that we’ve exported for use with C.

Of course it’s trivial to do the reverse and import C functions for use in the Go app, which I will leave as an exercise for the reader (look to the cgo docs as it has a number of examples doing this). One could easily make the cache provider a call out to another 0mq server from the Go app.

To prove this works, throw together a separate app (in Go, C, .NET, whatever) that acts as a client to this 0mq request/response server.

Structures in Go are laid out in memory identically to the same in C, allowing for potential zero copy use on either side (made possible given that addresses of data are fixed in Go — it is not a compacting GC, and doesn’t need the normal pinning functionality or expensive boundary crossing often seen in GC VMs like Java or .NET). There are concerns regarding who frees memory, and the lifetime of passed objects that *are* bound to be GCs, but those are topics for another time.

EDIT: Note that some members of the Go team are now (2014-09-04) talking about adding a compacting GC, and instead of discussing pin/unpin functionality, they’re pursuing dogma and wholesale contemplating blocking the sharing of pointers with C. This would be disastrously ill-conceived, but just had to point that out as a risk.

This is a trivial example, and is not intended to demonstrate Go best practices, nor is it necessarily a good example where you should add a second, less-safe language to your solution (in the example I used in the previous entry, I detailed how I use Go in one project for orchestration and communications, and very tight, heavily-vectorized C code for computations), but I think it cleanly and easily demonstrates just how easy it is to leverage C (and some of the incredible C/C++ benefits, like automatic vectorization as I discussed in my prior entry). While I linked to the 0mq library, of course you can build your own libraries as well for the C side of the equation.

I found it fascinating how simple Go made this and significantly much more complex mixed-language tasks, and thought it might prove useful for someone else.