Mumblings and Grumblings on Computers

Do The Easy Thing First

Out in the world, I see a lot of software teams falling into an insidious trap. They don't take care of the "Easy" things first. It shouldn't be surprising that teams often do this. The easy things aren't necessarily fun or interesting. No one writes blog posts about "Fixing builds when they are broken" or "Eliminating flaky tests". It's not a post that would get a lot of upvotes on Hacker News. But if you don't take care of the "Easy" things first, the hard problems become extremely difficult to solve. In fact, this makes for an interesting parallel to a model from the social sciences.

In 1943, psychologist Abraham Maslow described a "Hierarchy of Needs" in his paper "A Theory of Human Motivation". He described different levels of human need that must be met before higher levels of need can be met. At the bottom are the simple physiological needs; humans are not capable of greater achievement if they are hungry, tired, or thirsty.1 2 This is an interesting framework and one which could yield benefits if applied by software leaders.

A similar hierarchy of needs can be applied to the practices of a software team. This is not entirely a novel idea. The "Joel Test"3 is an example of an enumeration of a hierarchy. I don't agree with every element of the list and think that some of the elements on his list are more important than others. While it is not presented as a hierarchy, it satisfies my made-up criteria. The Joel test's 12 questions are:

  1. Do you use source control?
  2. Can you make a build in one step?
  3. Do you make daily builds?
  4. Do you have a bug database?
  5. Do you fix bugs before writing new code?
  6. Do you have an up-to-date schedule?
  7. Do you have a spec?
  8. Do programmers have quiet working conditions?
  9. Do you use the best tools money can buy?
  10. Do you have testers?
  11. Do new candidates write code during their interview?
  12. Do you do hallway usability testing?

Questions 8 and 9 are very basic level needs. If a developer isn't able to do the most basic parts of her work: the thinking and typing, she won't be able to do anything more complex. While questions 2 and 3 are higher level needs. I'm not sure that question 11 really fits into this framework, but the questions are still relevant. I won't propose a specific hierarchy myself. It's a complex question to answer which will vary across teams and settings. Additionally, there would be a different hierarchy for a team and for individual contributors. I will however say that testing, specifically reliable tests, and reliable deployment should always be at the most basic levels.

It's certainly fun to try to map the various techniques, skills and needs of software teams to Maslow's hierarchy, but this is more than just an interesting mental exercise. I believe leaders of software teams should use this model in setting priorities and guiding their teammates. For example, don't get bogged down in conversations about architecture if your builds aren't reliable. Further, don't adopt an architecture that will make it much harder to meet a lower level's needs. Or positively, consider adopting technologies or architectures that allow teams to more easily meet the lower level needs.

Additionally, leaders of software teams can use this hierarchy to inform the guidance they give to their team members. If a member of your team comes to you and asks how they can improve, begin by taking stock of the lower levels of the hierarchy. If they do not have a mastery or understanding of the lowest levels, ask them to start there. Specifically, don't tell them to read books about Service-Oriented architecture if they don't understand how their code is deployed to production. A coworker of mine wrote a blog post about "Good Software Developers"4, and while he does not discuss the idea of a hierarchy, I think the developer he describes is one who has mastered the lower levels of a software hierarchy.

In addition to directing the development of members of a team, a leader should also use this hierarchy to guide their team's priorities, i.e. don't adopt a complex architecture, if you are not able to test and deploy your code consistently. A leader of a software team should focus on the basics, once the team is executing the basics well, they have the foundation to solve more difficult problems.

  1. Maslow's Hierarchy of Needs

  2. Maslow's model, while very compelling at first look, does not have much empirical data to support it. So while it makes an interesting foundation to base our discussion upon, take it with a grain of salt.

  3. The Joel Test

  4. Drew Olson - Good Software Developers

Let the Design Decision Stand

Over the last five years I have been a part of a rapidly growing software development team. During this period, the team has grown from fewer than 20 developers to over 200. While this team has managed to remain effective there have been some growing pains. Of late, I have been thinking about some best practices for this team and have found some inspiration in an unlikely source.

In particular, the American legal system provides some principles and practices which can guide technical decision making on fast growing teams. At all times, teams must balance between remaining flexible and ensuring that changes are made intentionally. A team that is constantly re-evaluating every question of design will be ineffective and paralyzed by constant churn. And the converse situation is just as dangerous. If a team never re-examines their design decisions, they continue to make a bad design worse and worse.1

The legal system operates on the principle of precedent. This means that the previous decisions of other courts should be applied in subsequent related matters in other courts. Software teams should operate under a similar model. When intentional design decisions are made, it is important that they not be changed unintentionally.

To that end, when architectural decisions are made, they need to be documented. And further, the opinion needs to be documented. A team cannot rely on the presence of the person who made the decision in order to understand its purpose. Michael Nygard described some concrete steps to document these sorts of decisions on the Relevance blog in 2011.2 3 He proposes keeping Architectural Design Records (ADR) within your codebase to store information about decisions that have been made. I think this process is an excellent starting point for the sort of deliberate, precedent-based decision making I'm arguing for.4

Within the legal system, it is still possible to break with precedent. This is generally done by the Supreme Court in questions of Constitutionality. Because the burden to modify the Constitution is so high, the court is able to overturn existing precedent. Many of the highest profile decisions of the court are like this. Additionally, the court is able to make decisions without establishing precedent. And the court uses opinions to attempt to clarify their intent in making decisions and establishing precedent.

To follow these principles within the context of a software team means that there must be processes in place to re-interpret or amend an ADR as necessary. A strict constructionist interpretation of ADRs will lead to heartbreak. Additionally, it is important to be able to identify decisions that are made that do not mean that a precedent is being set.

Finally, assuming you are convinced of the value of this approach, the question remains of how to actually begin to put this approach into place. Obviously, you will not begin with a huge body of existing documents. So instead, an engineer or team that is looking to make a change in design and wants to establish a new precedent should begin with a good faith effort to justify the current design. In effect, backfill the ADR for the existing design and then amend it or re-interpret it to reflect the proposed new design. Concretely, people in technical leadership positions will need to accept the responsibility of monitoring the usage and adoption of this process. For example, they should be engaging in code review keeping these decisions in mind. Additionally, it is their responsibility to ensure that the contents of the decision documents are understood and socialized.



  3. A former coworker pointed me to this library for automating working with ADRs. Thanks phinze.

  4. Whether teams implement this process using ADRs as described by Nygard or something else is an implementation detail. If you disagree with that particular format, feel free to read thing you like better for the rest of the post.

What is Tensorflow?

In November 2015, Google released Tensorflow and by now it has become extremely high profile. At this point, most people in tech have likely heard about it and think to themselves "Tensorflow? That's the Google library for machine learning, right". But those of us, like me, who aren't involved in machine learning likely don't know much more than that.

Recently, as part of a project exploring interesting programming techniques to build programs that play the card game Bridge, I have been starting to learn about machine learning and Tensorflow. In this post, I will share some of the insight I've gained so far.

I think this is a relevant analogy to use to describe Tensorflow: Netflix is to AWS as Machine Learning is to Tensorflow. AWS works great for Netflix and Netflix is the most high profile application of it, but AWS is a way more general purpose tool. Similarly, Tensorflow works great for machine intelligence. The Tensorflow website itself acknowledges this saying in the header "TensorFlow is an Open Source Software Library for Machine Intelligence" but in the larger explanation of the library, the website says "TensorFlow™ is an open source software library for numerical computation using data flow graphs." Tensorflow is an excellent library for performing linear-algebra based computation. (It just so happens that one of the most common applications of this sort of computation is machine learning.)

When working with Tensorflow, there are two steps. The first step is the responsibility of the programmer. In this step, the programmer builds a computation graph. Using the Python or C/C++ APIs, he or she initializes nodes describing constants, variables, calculations, etc. In the second step, Tensorflow processes the graph and evaluates results. As an example, see below the code to multiply two numbers using Tensorflow.

import tensorflow as tf  
x1 = tf.constant(4.0)  
x2 = tf.constant(3.0)  
result = tf.mul(x1, x2)  
sess = tf.Session()  
## 12.000

In the sample we build up a computation graph, declaring two nodes that are constants, and a third node that is the result of multiplying the two constants. We then create a Tensorflow session and run the computation.

Alternately we could declare one of the nodes as a variable, which instructs Tensorflow that the node can change value. This would be useful for example if we were instructing Tensorflow to run an optimization on a computation.

import tensorflow as tf  
x1 = tf.Variable(4.0)  
x2 = tf.Variable(3.0)  
result = tf.mul(x1, x2)  
sess = tf.Session()  
init = tf.initialize_all_variables()  
## 12.000

Because the entire computation graph is built up at once, and because mutable vs immutable data is explicitly declared, Tensorflow is able to perform the computation very efficiently. It can distribute the computation across multiple nodes (the library uses Google's Protobuf library to facilitate this). It can also perform the computation on GPUs using CUDA.

I've found the small number of things I've done so far on Tensorflow to be pretty intuitive, but I definitely have not pushed myself very far. I will continue to use it as part of my Bridge project, and will report further as I learn more.


gen_fsm in Elixir

gen_fsm is a powerful behavior exposed in OTP. It is no longer exposed as part of the Elixir StdLib, but using gen_fsm directly from Elixir is possible and pretty easy to do. I haven't been able to find any good example code for this and thought it might be helpful.

As an example, I'll implement a simple switch which begins in an off state and enters an on state after receiving enough events. To do this, we need to implement an init function, and functions for the two states: off and on.

defmodule Switch do  
  def start_link(n \\ 10, opts \\ []) do
    :gen_fsm.start_link(__MODULE__, [n], opts)

  def init([n]) do
    {:ok, :off, %{n: n, received: 0}}    

  def off({:event}, state) do
    if state.received + 1 > state.n do
      {:next_state, :on, %{state | received: state.received + 1}}
      {:next_state, :off, %{state | received: state.received + 1}}  

  def on(_, state) do
    IO.puts "Turned on"
    {:next_state, :on, state}

For further reading on using gen_fsm, this chapter in Learn You Some Erlang is a good starting point.

Smell is A Smell

If you have worked in the software industry for any amount of time, you have likely heard something referred to as a "Smell". Whether it be a piece of code, a development practice, an organizational practice, or really anything, "Smell" is the easiest criticism for a software developer to level against a thing he/she does not like. I want to discuss the origin of this usage, why I find it overused, and how to improve the situation.

According to Martin Fowler, "Code Smell" "was first coined by Kent Beck while helping with [Fowler's] Refactoring book".1 Fowler defines "Code Smell" as "a surface indication that usually corresponds to a deeper problem in the system." He continues on to note that a "Smell" should be something that is quick to spot. It can point to a deeper problem in a code base, but does not necessarily. It is not sufficient to identify the "Smell", but also to look for it's root cause and determine if the "Smell" is indicative of a greater problem.

What makes the presentation of "Code Smells" in Refactoring so effective is that each "Smell" is followed by discussion of the larger problems that it is likely indicative of. Additionally, the author presents refactorings that can be used to fight the "Smell". For example, the "Middle Man" is a "Smell" that is recognized when a object only delegates to another. This "Smell" suggests that the delegating object provides no value and should be removed. Three refactorings are suggested to alleviate the problem.

The author of Refactoring relies on a wealth of experience in presenting "Code Smells". When a "Code Smell" is identified in Refactoring, it means a great deal more than "looks bad at first glance". It instead means "We have seen this easy-to-identify problem many times before and have determined that it is often related to this specific underlying issue; here are specific tools to fix the problem."

Outside of the realm of "Code Smells", we do not have these same foundations. For that reason and others, I think we should consider the over-usage of "Smell" a "Smell". It meets all the criteria: it is very easy to recognize and can indicate a problem where-in engineers are too fast to criticize without identifying root causes and possible solutions.

I believe that it is far too easy to decree something a "Smell" and simply move on. The burden of proof for somethings "Smell"-worthiness is too low. I believe that it is a way to criticize without having any need to supply justification. I believe that we can do better than decreeing something a "Smell" and then dropping it.

Some might ask, "What is the harm in this beyond annoying you personally?". I believe that the overuse of "Smell" leads to a culture of negativity and un-constructive criticism. I believe that stopping at declaring something a "Smell" without further investigation prevents people from learning and understanding the process of making difficult tradeoffs in designing systems, processes, and organizations. Finally, I believe that it leads to faulty decision making. The weight of "Smell"-accusations can add up to such an extent that well considered trade-offs are ignored.

So what should one do instead. If you identify a "Smell", don't just say "This might be a smell" and leave it there. Commit to examining the source of the "Smell" and determine whether or not it's an actual problem. If it's indicative of a problem articulate what that problem is and begin to work on a solution. Record this organizational/process "Smell", note what larger problem it was indicative of, and document its solution.

If we as an industry commit to this kind of close inspection of our problems, we will all be better off.