Posts

Leader election, in distributed computing, is the process of designating a single process as the organizer of some task distributed among several computers (nodes). Before the task has began, all network nodes are either unaware which node will serve as the “leader” (or coordinator) of the task, or unable to communicate with the current coordinator. After a leader election algorithm has been run, however, each node throughout the network recognizes a particular, unique node as the task leader....

I like to write code, build, and release tooling, for my brain at 3 AM. No surprises, no clever bits to trip me up. Lately I’ve been playing with GitHub Actions to build and deploy some of my public projects. It took a little while to figure out something that I could copy and paste between them, and I’d like to share what I’ve managed to come up with. Build Actions Committed to your project as ....

As you can tell, I’ve switched this blog from Jekyll to Hugo since its entire toolchain comes in a single downloadable binary. I don’t need bells and whistles, and the Paper theme makes it look like I still know how to make pretty things. While Hugo has the hugo new generator, and it works really well, I wanted to preserve the hugo import jekyll migrated filename pattern. Time for a new skeleton script, this time in ruby because I used bash in the last one....

I’ve been playing around with using an in-process redis using miniredis as the backing store for a service that relies on go-workers2 for background processing. You can find the code in my example-miniredis project on GitHub. While miniredis was created as something to be only used in unit tests, this may be useful in running a service that normally requires a redis in a totally standalone mode. I view such a standalone mode as critical for a good development experience in creating integrations against a service, since you can run the service locally without any of its downstream dependencies and still expect to have it respond sensibly....

It has taken me far too long to figure out all the minute details of how to apply authentication to gRPC-based services, how to do that using the HTTP/JSON grpc-gateway, and then how to run a gRPC server and a grpc-gateway side-by-side using the same service instance in a way that makes sense to me. There’s a surprising amount of detail that is not documented and there are a few gotchas for the unwary traveller....

My day job currently includes writing and maintaining golang services. It’s not a bad language, and it certainly forces you to understand that everything will fail, far more obviously than Java’s checked exceptions. I’m not going to argue about the utility of checking/returning error instances, but I do find myself writing the same code over and over again, especially when I’m handling any SQL operations. So I’ve come up with a couple of templates to remove a bunch of the boilerplate code for transactions and for looping over SQL query result rows....

I’ve recently been involved with integration and administration of an OpenLDAP cluster, and was a little dismayed at the state of monitoring of OpenLDAP. You can certainly get good machine metrics using the Prometheus Node Exporter, but there wasn’t really anything good for OpenLDAP-specific metrics aside from an interesting project based on python. This exporter had some really good ideas, but I baulked at installing and running twisted python, irrespective how good it is, on slapd nodes....

I use Jekyll to create this blog, but creating new posts has been a bit of a pain, since you have to create the post files by yourself. To make it easier, here’s my script that generates a skeleton post file: #!/bin/bash set -euo pipefail if [[ $# -eq 0 ]]; then TITLE="-h" else TITLE="$*" fi if [[ "${TITLE}" == "-h" ]]; then echo "Generate a new Jekyll post" echo "Usage: ....

Starting a HTTP server in python to serve files from a directory is a reasonably well-known one-liner. In python 2.x it is: python -m SimpleHTTPServer 8080 In python 3.x it is: python -m http.server 8080 But how do you something similar for HTTPS? Here’s a solution, which unfortunately is larger than one line: #!/usr/bin/python import BaseHTTPServer, SimpleHTTPServer import ssl httpd = BaseHTTPServer.HTTPServer(('0.0.0.0', 8443), SimpleHTTPServer.SimpleHTTPRequestHandler) httpd.socket = ssl.wrap_socket(httpd.socket, certfile='./certs_and_key.pem', server_side=True) httpd.serve_forever() Save that as an executable file, combine your PEM cert chain and key into a single file, and off you go!...

Everyone has a backup script that takes a tarball/zipfile/etc and uploads it somewhere for safe-keeping. In a lot of places where I’ve worked, the “somewhere” winds up being an Amazon S3 bucket (or lately a DigitalOcean Space). These scripts are lovingly crafted and sometimes quite clever, using the aws cli or s3cmd, or something custom. What I wanted was to encode my process for encrypting a backup and sending that backup to an S3 bucket....