Open source at Infinity Works in 2018

We use a lot of open source software. The Linux operating systems on (some of) our computers, the text editors we use to write software (e.g. VS Code, vim), the programming languages we use (e.g. Go, .NET, PHP, Python, Java) the libraries and utilities we use and the products built themselves: Web servers, databases and other tools.

I’m not sure it’s easy to create software using only proprietary tools any more. Even if you used Visual Studio on Windows, a lot of .NET code is open source, so you’d probably end up using an open source library.

It’s easy to take open source for granted. As software engineers, we’re now used to being able to look at the code of a product or tool we’re using to understand what it’s doing and even be able to propose changes going on, but it wasn’t always that way.

Successful open source projects require effort from the maintainers who provide their time (often for free) and from the wider community in the form of testing, issues, pull requests and promotion.

I decided to have a look at what the people at Infinity Works had been contributing in 2018. To make it easier for me, I limited it to Github and I tried to remove our “day jobs” (the work we get paid to do for our clients). Even though we may do open source work for clients, I was interested in other contributions.

There’s 139 people in our Github organisation. Here’s what I found.

Issues

We created 71 issues. An issue can be a bug report, a feature request or even a request for help. The number surprised me because it’s so low, but it’s probably accurate.

People use issues in lots of ways. Some raise issues within repositories now owned by them, but others are using them as placeholders for features, e.g. raising an issue in one of their repos to remind them to do something.

The top issue creator was wojtek-oledzki ([0]). It looks like he’s working on a project called [1] which is about using Ansible to setup a lot of modern infrastructure like Kafka and is using issues to track work.

• https://github.com/wojtek-oledzki [0]

• https://github.com/sansible [1]

YOU54F ([2]) is creating a lot of issues and fixes around Pact (consumer-driven contract testing). dvejmz ([3]) is finding a lot of issues and missing files and AdeOpe ([4]) has been sharing his scripts for getting OAuth up and running with Cognito.

• https://github.com/YOU54F [2]

• https://github.com/dvejmz [3]

• https://github.com/AdeOpe [4]

I guess overall it’s good news, it means that most people probably didn’t have any issues with open source software that they needed help with via Github, since the top 10 people raised 50 of the 71 issues.

Pull requests

Pull Requests are the next level of engagment. It’s a contribution of changes to software. They’re tricky.

Sometimes, your pull request gets merged, sometimes it doesn’t. It depends on lots of factors:

• Whether your change is well written / matches the project style.
• Whether the maintainers want your change / it matches the project goals.
• Whether the maintainers are busy or don’t care about the project any more.

I’ve been on both sides here. I spent a long time on a pull request to a text editor adding autocomplete (a feature I really wanted - [5]), but the maintainer never found the time to merge it. Also, I feel a bit guilty about a pull request in a project I started (a JSON Schema to Go generator).

• https://github.com/zyedidia/micro/pull/977 [5]

I wrote it because I needed it for a project, I’m not a fan of JSON schema or anything. It’s a big change and I’m happy if the users of the tool want to merge it, but I’m not really an expert in JSON schema, and don’t particularly want to spend my leisure time thinking about it, so it’s been sat there for a while until I have a rainy day to think about it.

So… how many pull requests have we raised but are sat there unmerged?

I’m sad to say that it’s 54 pull requests just withering away on branches. The people who wasted the most time on unloved code might be JamesSheard ([6]) and MikeSilverstone ([7]) with six pull requests each. The good news is that it seems to be part of some training they were doing, so hopefully they learned something.

• https://github.com/JamesSheard [6]

• https://github.com/MikeSilverstone [7]

Apart from that, we had ## 161 merged, they’re pretty diverse, but from what I saw, mostly small changes to various libraries and Websites.

Quantity of repositories

So who has open sourced lots of personal projects? cindyjialiu ([8]) is in the clear lead here, having done a lot of proof-of-concept and learning work across a wide range of programming languages in the last year.

• https://github.com/cindyjialiu [8]

I’m next, in the last year I’ve made a number of small utilities and libraries for Go:

• Go: Dates which support custom JSON formatting - https://github.com/a-h/date
• Go: ordered maps / sets - https://github.com/a-h/setof and https://github.com/a-h/mapof
• Some Serverless security tools for analysing Github repos and keeping up-to-date with the NIST National Vulnerability Database - https://github.com/a-h/nvdnotifier and https://github.com/a-h/watchman
• A command-line file search tool (a bit like grep) - https://github.com/a-h/search
• Some machine learning algorithms - https://github.com/a-h/ml
• Go: middleware to hide content behind Google domain authentication (to have a website private to a Google org) - https://github.com/a-h/gauthmiddleware
• Go: a library to extract path paramaters from URLs to prevent you from needing to reference your HTTP router in your HTTP handlers (having to reference your router couples your handlers to your router too tightly for me) - https://github.com/a-h/pathvars
• Go: a cache designed for Serverless applications that uses Kinesis to update multiple nodes as data changes - https://github.com/a-h/scache/
• Go: tools for writing parsers - https://github.com/a-h/lexical
• An app for tracking your phone’s mobile connectivity on train journeys (the GPS resolution wasn’t really good enough) - https://github.com/a-h/Connect

d-baranowski’s ([9]) 2018 interests seem to be javascript visualisations, while benbrunton’s ([10]) been doing a bit of rust and some socket.io stuff and steinfletcher ([11]) looks to be doing something interesting with sequence diagrams and Go.

• https://github.com/d-baranowski [9]

• https://github.com/benbrunton [10]

• https://github.com/steinfletcher [11]

All together, we’ve updated 280 of our personal repos in the last year.

Stars

OK, the popularity contest. I’ve counted stars based on only counting stars from repos updated in the last year. It’s stars for people’s personal repos only.

Rhymond ([12]) is most popular with 654 stars this year, mostly with the [13] library.

• https://github.com/Rhymond [12]

• https://github.com/Rhymond/go-money [13]

Next most popular is JamesBarwell ([14]) with repos with 467 stars updated this year. His Node.JS library for working with Raspberry Pi ([15]) GPIO ports ([16]) is really popular.

• https://github.com/JamesBarwell [14]

• https://www.raspberrypi.org/ [15]

• https://github.com/JamesBarwell/rpi-gpio.js [16]

I’m next most popular for the JSON Schema to Go struct thing (which is due a rewrite to use a neat Go JSON schema library I found).

What else?

I can’t talk about community without mentioning the real-world meetups that Infinity Works people help out with. We run or sponsor community events, not because we have to, but because we enjoy it.

We run a couple in Manchester that I really like.

Manchester: DevOps Battle Royale ([17]) - short talks followed by group discussions, a technological victor must emerge.

• https://www.meetup.com/The-DevOps-Battle-Royale/ [17]

Manchester: Infinity Works 101 sessions ([18]) - free training in cutting-edge tech.

• https://www.meetup.com/Infinity-Works-101-Sessions/ [18]

I also like CodeUp. I tend to go to the one in Leeds, which is sponsored by Infinity Works ([19]) but others in the team go to the Manchester one (sponsored by Code Computer Love) ([20]).

• https://www.meetup.com/CodeUp-Leeds/ [19]

• https://www.meetup.com/CodeUpManchester/ [20]

I used to love a bit of Leeds DevOps (sponsored by Infinity Works) even though I haven’t been to one since I’ve been working in Manchester and I’ve been to one GoLang Manchester ([21]) but it always seems like it clashes with something else for me.

• https://www.meetup.com/golang-mcr/ [21]

I think it’s a great thing that people want to spend their time to learn from others and share what they know.

The community at Infinity Works also stumped up £300 last year to donate to an open source project out of their own pockets (which was then doubled to £600 by Infinity Works).

How did I find this out?

I wrote some code in Go at [22] which uses the Github GraphQL API ([23]) to collate the data. If you’re interested in using the Github API, it’s probably a good script to look at anyway.

• https://github.com/a-h/openanalysis [22]

• https://developer.github.com/v4/ [23]

You’ll need to pass it a Github API token if you want to run it yourself ([24]).

• https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/ [24]

The first part of it creates a JSON file containing the data for each Github user in the users.json file, while the sum script creates the summary data.

So feel free to run it against yourself and I supposed you should submit a pull request if you think the tool could be improved or drop me a star so I feature more highly next year.