If you’re a developer who has done TDD, you probably know what a big difference fast running tests make. By isolating your code and limiting dependencies, you can get quicker feedback which allows you to make progress more quickly. At HubTran, we’re using that same concept of rapid feedback as a way to verify customer value in our features.
Last week I wrote about how we’re increasingly using self selection in our hiring process. In that post, I mentioned that we might release features that require manual work in order to get them out the door faster. Today, I want to expand on that a little.
Recently, we started serving a very large customer. That customer had a legacy process that they wanted us to support. Due to their size, we decided this was a good idea. Their legacy process involves uploading some documents along with a spreadsheet of supporting data. This spreadsheet is generated by their clients, so as you might imagine, it’s not always in exactly the right format and doesn’t always contain the right data.
When we first started thinking about how much work it would be to handle this new flow, we were nervous about the number of things that could go wrong. We came up with dozens of ways that things might fail. We knew that handling every case would take a long time. At the same time, we had some fundamental risk in this process. While we were confident we could make this new flow work, we knew we might be making some initial bad assumptions. To that end, we decided to release our first version of the flow with almost no error handling. If something went wrong, we would need to manually fix the data to allow our customer to move on. Because they were already planning on an incremental rollout, we knew this wouldn’t overwhelm our team. We talked to our customer, and they were on board as well.
When we launched, we found that about 40% to 50% of uploads had problems that required manual work. Clearly this was a lot of manual work! Once we had some data on the types of manual fies we needed to make, we would build in appropriate error handling. If we were successful in reducing the number of uploads that required manual intervention, we would ask our customer to add more volume. If we didn’t, then we made more fixes. Within a few months we were handling their entire volume while only making a fix or two per week.
By waiting to build error handling until after the initial release, we were able to get this feature in the hands of our customer sooner. This turned out to be a win/win. Our customer got access to a new feature sooner, and we were able to reduce our risk by validating our assumptions quickly. As an added bonus, we got real world feedback on what types of errors were most likely to occur.
This type of rollout isn’t right for every circumstance. If we weren’t able to control the adoption of the feature, it would be easy to be quickly overwhelmed by manual work. In this case, however, we were able to reduce risk while releasing a feature more quickly.
