Over the last few years, Michael Niessner and I have spent a lot of time looking at patterns that predict the success and failure of both software products and software engineers. We’ve done this both to improve our success rate at HubTran, but also to help us understand how to hire better software developers. While we’re still working on finding the secret to success and better hiring, we’ve found at least one combination of traits that is a sure sign that failure is around the corner. If we see Assumptions and Confidence, we know we’re looking at trouble.
Now, that might be surprising to those that know me. I’m pretty sure I’ve never been accused of lacking confidence. While I may be very (over)confident in my day to day life, I try hard to make sure that doesn’t bleed through into my software and product life. As a person who is quite confident, I take special care to think about my assumptions. For example, let’s say I’m building a system that allows you to create and process invoices for truckload shipments. To store the invoice, I can add an invoice_id column to my Load model. At this point, I try to ask myself what assumptions am I making and what is the impact if these assumptions are incorrect. In this case, I’m assuming that there can only be one invoice on each truckload. Is that a good assumption? It turns out it isn’t. If you never stopped to call out this assumption and do the work to learn if it is valid, you wouldn’t know that. Instead, you would keep building software that wouldn’t work once it was faced with real world scenarios.
We’ve seen quite a few developers make this mistake. When asked what happened, we often hear “The requirements didn’t specify there wouldn’t a single invoice on each truckload.” While that’s true, the requirements also didn’t specify there would be exactly a single invoice on each truckload either. To me, knowing what assumptions you are making in translating your view of the world into code is a critical skill to have.
Knowing what assumptions you’re making isn’t sufficient to do things right. You also need to verify that your assumptions are valid. This could mean talking to a domain expert. It could instead mean looking through data for counterexamples. The best developers we know are experts at understanding their assumptions and knowing which assumptions are likely to cause problems down the road. They then spend time worrying about high risk or high cost assumptions and worry less about non-critical or easy to change assumptions.
These same developers are experts at finding cheap and easy ways to test their assumptions.
Kent Back has made a career in thinking about the assumptions that underlie the status quo and asking what happens if they are wrong. He discusses this in the Unlearn podcast by Barry O’Reilly. By using thought experiments to decide what the outcome of an assumption being incorrect would be, Kent can quickly test ideas to decide if they are interesting enough to experiment with.
We will often do something similar at HubTran. We want to perform sensitivity analysis around our assumptions. Recently, we had a member of our team mention that a certain step in our process was time consuming. He was thinking about whether we could make it faster and had spent several days working through some ideas for enhancements. Before we spent developer time on these changes, however, we wanted to test the assumption that this issue was causing our customers real pain. When looking at real world usage, it turned out that this flow was hit only a few percent of the time. Our team member was often tasked with training new accounts before HubTran had finished learning. In his experience, this was a real problem. Looking at data, it turns out to not be much of an issue. If we hadn’t checked that assumption, we would have spent weeks optimizing a very minor issue.
I hope it’s clear that I’m not saying you shouldn’t be confident, or that you shouldn’t make assumptions. Both are critical to building software. Instead, I hope you will consider when it makes sense to validate the assumptions you are confident in. As we’ve seen, the best developers have mastered this skill. With practice, we believe you can too.
