Hi

Iʼm Murray and I want to tell you about a contribution I made to rubygems.

First some background:

I work for Unboxed which is an agency that specialises in helping our clients develop services and products using agile and lean techniques.

What this means for me is that when working with a client I spend some of my time talking to them to define what they want their products and services to be…

…some of my time writing on cards to clarify these discussions and break them up into discrete chunks of work…

…some of my time building software (usually a rails app) to provide the functionality described on these cards…

…and some of my time showing the client the software we’ve built and working together to explore what the next iteration is that will get us closer to their ultimate goal for the product.

It’s rewarding working with people to create something, but long client projects can be pretty intense and tiring. Thereʼs always another feature to build, always another stakeholder to convince, and always another prototype to be tested, learned from, and converted into production code.

In between client projects I like to take a few days to decompress. Obviously this can mean going on holiday and eating gelato in an Italian Piazza, but what I really mean is finding something to do at work that’s less taxing than client projects. I do things like upgrade the OS on my laptop or investigate new dev tools Iʼve read about but not had a chance to play with yet.

I also look at our internal projects and open-source tools to see if there’s anything I can do to improve them. This usually means doing maintenance upgrades; you know: bump a dependency version here, resolve a deprecation warning there, upgrade to new APIs … everywhere.

This is the perfect kind of work to do for these breaks. You get to learn about changes to familiar dependencies in a low-risk environment, and you donʼt usually have to think too much; test failures, runtime error messages, and upgrade documentation (when it exists) guide you through the whole process.

Itʼs on one of these breaks way back in September 2014 where my story begins:

Iʼd already tidied up my laptop and dev tools was looking for something else to do.

We have a gem called “be_valid_asset” which provides rspec matchers that check if your HTML, CSS, or feeds are valid according to the w3c specifications. The gem provides these matchers to work with any version of rspec, but itʼs own specs are written in rspec 2.

I decided to upgrade those specs to use rspec 3 and at the same time convert them from the old should syntax to the new expect syntax.

This is an really easy process because of the excellent transpec tool by Yuji Nakayama. The first time I did a rspec 2 and should to rspec 3 and expect upgrade I did it by hand because I didnʼt know about transpec. It took ages and I almost gave up. The next time? I used transpec and it was a breeze.

Transpec is amazing and you should use it.

So, after running transpec and tidying up a couple of other things to make sure that the matchers still worked with earlier versions of rspec even though we now tested them with rspec 3, I was ready to release the gem. I bumped the version number and ran the gem build command to produce the new version.

This is where my problems began; rubygems wouldn’t build our gem and spat out this error message.

Our gemspec said that you could use the gem with any version of rspec (the runtime dependency), but if you wanted to develop the gem you needed rspec 3 or greater (the development dependency).

The error tells me that I canʼt specify rspec twice and should combine the separate runtime and development dependencies into one runtime dependency ultimately saying you needed rspec 3 or greater, which isnʼt what we wanted - the matchers can be used with earlier versions of rspec. This seemed odd to me as I only changed a single character in the gemspec; swapping a 2 for a 3 in the development dependency for rspec ¹.

So what changed?

First I found the code that raises the error in the rubygems source to check if it was raising as it should (it was). Then I read more of the surrounding code and the project documentation to see if there was a different way to express my dependencies (there wasnʼt). Finally I did some code-archaeology to find out when that part of rubygems had changed and see if I could work out why.

I found out that Rubygems 2.2 introduced the error I was seeing, and weʼd last released the gem using an older version which explained why it worked before. This version of rubygems would refuse to build a gem if the gemspec listed a dependency twice, I think as a way to get maintainers to tidy up their gemspecs.

I didn’t have the full context on why they’d made the change but it seemed reasonable, what didn’t seem reasonable was not taking the type of the dependency (runtime vs. development) into account. Surely lots of gems that provide tools for testing would have similar dependency schemes to mine, so I filed a bug.

I described the error I saw, what I expected to see, and gave an outline of what I thought the solution might be.

Even though I’d done some research and raised an issue I wasnʼt too sure if what Iʼd found was really a bug. I really wanted to release our gem and get it off my todo-list, so I looked for a workaround.

The source for be_valid_asset uses bundler so I was able to move the development dependency out of the gemspec and into the Gemfile. Rubygems only looks at the gemspec when building the gem so this avoids the issue entirely. It’s annoying that rubygems doesn’t know about the development dependency on rspec 3, but anyone coming to the code will soon find it in the Gemfile.

I committed this change and included a note about what I’d found along with a link to the issue I’d raised so I could come back to it later.

Workaround done, I was able to release the gem.

Back to the issue.

A rubygems maintainer, Eric Hodel, commented on the issue to confirm that what Iʼd found was a bug, and that my suggested fix sounded correct.

I worked on it a bit until I had some code to push up, and turned the issue into a PR with that code attached².

I had a couple of questions about my approach, particularly around going a little bit further with my solution so I asked these in a comment at the same time.

Eric replied thanking me for the code, said not to worry about the other issue, and said heʼd merge it when he had some time.

Another Rubygems maintainer, Daniel Berger, merged the PR, and added a note to the changlog about it.

This got bundled up into version 2.5 of rubygems - which I was pretty excited about - I use rubygems nearly every day and now itʼs got my name on it!

So that’s my story. I tried to use a tool, found a bug, raised an issue, turned it into a PR, it got merged and I got my name into the list of Rubygems contributers. Apart from needless boasting, why’d I tell you all this?

The reason I told this story is that I think it has a few interesting points about open source contributions:

1. How we report issues
2. How we working around the issue
3. How we communicate with the maintainers

First up: reporting an issue.

When I found the issue, I immediately assumed rubygems was broken because I am perfect. However, before raising an issue accusing them of being idiots I took some time to investigate to find out where the fault really was. Iʼd have looked pretty foolish if the maintainers pointed out I hadnʼt even read the docs where it clearly says you canʼt do this.

By doing some research I was fairly sure that the error I’d encountered was caused by a real bug. I included all these details in the issue I raised. It’s a far from perfect bug report, but it had plenty of detail in it so the maintainers could understand what I was trying to do, the problem I encountered, and what I thought the solution might be.

I didnʼt jump straight to coding because even with my research I had no idea if what Iʼd suggested was actually how they’d want to fix it. It could be that the fix isnʼt the code solution Iʼd offered, but a change to the documentation that makes it clear you canʼt have duplicate runtime and development dependencies (even if you might want to).

The issue I raised ended with an offer to work on the solution if the maintainers thought it was a good idea. Iʼm telling them “hey I found this thing, and if you agree itʼs a problem Iʼll work to fix it” - Iʼm not just adding to their workload.

Second: workarounds.

Most of my open-source contributions come from finding something that can be improved about a tool I’m using, while I’m trying to use it to do something else. It’s a bad place to be if your client work can’t continue because it relies on a bug being fixed in upstream code. Even if the maintainers agree that the issue is a bug, thereʼs no guarantee theyʼd give it any kind of priority that aligns to when I want it done. Their roadmap is certainly not my roadmap.

Knowing this was a possibility I came up with a workaround for be_valid_asset that let me actually release it.

This turned out to be pretty sensible because what I didnʼt mention before is that it took a day between me raising the issue and Eric saying it looked legit, then a couple of days for me to produce the code and submit it.

Eric said heʼd look at it later that same day, which was great. But it also took 10 months between Eric saying that and Daniel merging it.

And then another 3 months until a version of rubygems with my change was actually released.

Releasing a new version of be_valid_asset wasn’t very important, but imagine this was production code that you need to release quickly. A few days delay could be a real problem, and months almost certainly would be!

Itʼs worth saying that I didnʼt commit my workaround a week or a month into this process. I did it straight after reporting the issue.

It took over a year from reporting the issue to there being a released version of the software that solves it. Because I worked around it though, I didnʼt really care. I just got a nice surprise the next year when a colleague mentioned my name was in the release notes for the new rubygems.

Which brings me finally to communicating with the maintainers.

What I showed in my story is all of the communication between me and the maintainers on this issue. I could have prompted them every so-often for an update, but I donʼt think that would been helpful to anyone.

Iʼm sure that at work weʼve all been in a situation where a client or colleague is continually asking you if something is done yet, and each time they ask you have to tell them itʼs not. These questions never help you get it done faster, and I doubt you enjoyed it. And yet this is exactly what weʼre doing to open-source maintainers when we leave “+1”, “me too”, or “is this project dead?” comments on issues.

Many open source maintainers are volunteers and we canʼt expect them to jump to it whenever we drop some new work in their lap.

Of course it can be frustrating to not get a response, but itʼs almost certainly not because the maintainers are ignoring you. Chances are theyʼre doing this in their spare time, and if their day job is a full-time developer itʼs not surprising when they donʼt want to come home and spend the rest of the day doing the exact same thing as they do at work.

In summary, when contributing to open source:

1. Raise an issue first to find out if the maintainers agree that what you want to do is something they’ll accept. If they donʼt agree youʼve not wasted any time on the hard part of actually fixing it.
2. Donʼt assume your issue will be accepted and worked on or your PR will be merged. Expect that it won’t be and come up with workarounds in any project that relies on the fix.
3. Donʼt hassle the maintainers for an update. Theyʼll get round to your issue at some point, and if they donʼt maybe they changed their mind about how appropriate the change is. That’s their prerogative.

Thanks for listening³.

Bye!

Here’s the credits for the photos I used from flickr (all creative-commons licensed).

• Binary code: https://www.flickr.com/photos/132889348@N07/20607150556/
• Rijksmuseum (3) - Amsterdam: https://www.flickr.com/photos/reisgekki/13192234543
• Card catalog: https://www.flickr.com/photos/mlibrary/7021368159
• Student using a microfiche reader, 1982: https://www.flickr.com/photos/cochranlibrary/14112708781

All other photos were mine, or from the Unboxed asset library.

The rubygems logo on the title slide comes from their logo asset library.