Why You Should Test Your Code

Bill Gathen

June 10, 2015

You may have heard people talk about testing their code. Another way to put it is writing specs. But what's that all about? To understand why we test, let's start with a simpler question. Take a look at the code snippet below. 

Is there something wrong with this code?

def multiply_by_two(n) 	
	n * 2 
end

Even if you don’t understand Ruby, you can probably guess that the behavior of the method matches the name, so I’d say it’s probably correct. (If you’re wondering why there isn’t an explicit ‘return’ statement the way there would be in other languages like Javascript, that’s because Ruby automatically returns the last expression processed, so we don’t need an explicit “return” statement.)

It looks good, but how can we be sure it works correctly? We need to run the code and see if the results match our expectations.

Ruby has a command-line tool called irb which creates a sandbox to interact with your code in real-time. irb is a “REPL” or Read-Eval-Print-Loop, and many popular languages supply one as well. For example, the node and python commands start REPLs -- for JavaScript and Python, respectively -- when you run them from the command-line without an argument.

You can start irb by going to your command-line and typing `irb`. Go ahead and give it a try! With irb, we can manually test this method with different values. If our code was in a file called math.rb, our irb session might look like this:

> irb 
2.2.0 :001 > require './math' 
=> true 
2.2.0 :002 > multiply_by_two 2 
=> 4 
2.2.0 :003 > multiply_by_two -2 
=> -4

The “require” command pulls in our file so we can work with it, then we call the method with any arguments we like. The return value appears on the next line after the fat arrow (=>). It appears to work with 2 and -2, so are we good?

2 and -2 are valid values, so we’ve tested the “happy path”. The happy path is when we get normal, expected inputs and generate normal, expected outputs. But what about the “sad path”? What if we forget to supply a value? What if what we supply isn’t a number? What if we supply a variable that hasn’t been assigned a value yet (Ruby defaults to nil)? Let’s try those out and see.

2.2.0 :004 > multiply_by_two 
ArgumentError: wrong number of arguments (0 for 1) 	
	from /Users/bill/dev/math.rb:1:in `multiply_by_two' 	
	from (irb):4 	from /Users/bill/.rvm/rubies/ruby-2.2.0/bin/irb:11:in `<main>' 
2.2.0 :005 > multiply_by_two "ruby" 
=> "rubyruby" 
2.2.0 :006 > multiply_by_two nil 
NoMethodError: undefined method `*' for nil:NilClass 	
	from /Users/bill/dev/math.rb:2:in `multiply_by_two' 	
	from (irb):6 	
	from /Users/bill/.rvm/rubies/ruby-2.2.0/bin/irb:11:in `<main>'

It looks like we have some problems after all. Let’s add some error-handling. Error-handling code deals with unexpected or invalid inputs and gives the program an alternate plan for dealing with them. That way, the code doesn’t just break.

def multiply_by_two(n = "no value supplied")
 	if (n.is_a?(Integer))
 		n * 2
 	else
 		raise ArgumentError.new("Please supply a valid number")
 	end
 end

The “no value supplied” will default our argument to something invalid if we forget to supply one. The if statement uses Ruby’s built-in .is_a? method to check if what we supplied is a valid Integer (a whole number). If it is, our code runs normally. But if it’s not, the “else” clause raises Ruby’s built-in ArgumentError with a useful message. Whoever calls this method will need to decide what to do when they receive the error.

Let’s test our code again to see if we did this right.

> irb 
2.2.0 :001 > require './math' 
=> true 
2.2.0 :002 > multiply_by_two 2 
=> 4 
2.2.0 :003 > multiply_by_two -2 
=> -4 
2.2.0 :004 > multiply_by_two 
ArgumentError: Please supply a valid number
 	from /Users/bill/dev/math.rb:3:in `multiply_by_two'
 	from (irb):4
 	from /Users/bill/.rvm/rubies/ruby-2.2.0/bin/irb:11:in `<main>' 
2.2.0 :005 > multiply_by_two "ruby" 
ArgumentError: Please supply a valid number
 	from /Users/bill/dev/math.rb:3:in `multiply_by_two'
 	from (irb):5
 	from /Users/bill/.rvm/rubies/ruby-2.2.0/bin/irb:11:in `<main>' 
2.2.0 :006 > multiply_by_two nil 
ArgumentError: Please supply a valid number
 	from /Users/bill/dev/math.rb:3:in `multiply_by_two'
 	from (irb):6
 	from /Users/bill/.rvm/rubies/ruby-2.2.0/bin/irb:11:in `<main>'

The tests for 2 and -2 continue to work, so we didn’t break our existing logic for valid values. The three error cases -- or sad paths -- we took, now all raise a consistent error with a helpful message.

So we’re done, right? Let’s ship it!

Not so fast. Using irb to work with code is pretty cool, but:

  • Do we really want to do this dance every time we make a change to this method?
  • How do we remember all the cases we tested?
  • How do we remember all the decisions we made about what was “correct” behavior in all these cases?
  • What if someone else touches the code and we want to know what behavior they added or broke?
  • What if someone else looks at this code and wonders if we checked this stuff?

This was a pretty simple example, but imagine going through this same testing process for an actual program with hundreds or thousands of methods to keep track of!

What if there was a way we -- or anyone else on our team -- could do all that testing automatically, just by running a single command?

Welcome to automated testing

Automated testing uses a testing “framework” that wraps around your code and interacts with it the same way we’ve been interacting with it through irb. Ruby has the RSpec and Minitest frameworks. JavaScript has Mocha and Jasmine. Your language likely has a selection as well: check this out.

Testing frameworks have several features that make automated testing much easier and more powerful than manual testing. They typically:

  • Run all tests (or a subset) with a single command, even if you have hundreds or thousands of them
  • Supply assertion logic to give a “thumbs-up/thumbs-down” result for each test scenario. If you configure your test with colors, this usually means that you’ll see green when the test passes, or red when it fails
  • Permanently document all the conditions you want to try, along with the behaviors you expect in response
  • Run blazingly-fast, executing the tests as quickly as your language can run, instead of as fast as you can type

If you’re using a tool like git, which you should be, then your automated tests get checked into source control alongside your main code, making it available to everyone who comes after, even if it’s you six months from now when you’ve forgotten everything this application was designed to do. It’s ok, we’ve all been there.

Here’s the automated version of the tests we did manually, this time in Ruby’s most-popular testing framework, RSpec:

# math_spec.rb 
require './math  

RSpec.describe ".multiply_by_two" do
 	let(:expected_error) { "Please supply a valid number" }

  	it "returns correct results for valid values" do
 		expect(multiply_by_two(2)).to eq(4)
 		expect(multiply_by_two(-2)).to eq(-4)
 		expect(multiply_by_two(“zero”)).to eq(0)
 	end

  	it "raises an error message for a missing argument" do
 		expect{ multiply_by_two() }.to raise_error(expected_error)
 	end

  	it "raises an error message for invalid arguments" do
 		expect{ multiply_by_two("ruby") }.to raise_error(expected_error)
 		expect{ multiply_by_two(nil) }.to raise_error(expected_error)
 	end
 end

Don’t worry if you don’t understand every character, but hopefully you get the idea. Each test scenario has an assertion -- in this case “expect” -- that says “if I run X, I expect Y”. If the result is anything other than what we expected, we get an error. Let’s run it now at the command line and see how we did:

> rspec math_spec.rb 
F..  

Failures:  

1) .multiply_by_two will handle valid values
 	Failure/Error: expect(multiply_by_two("zero")).to eq(0)
 	ArgumentError: 	Please supply a valid number
 	# ./math.rb:3:in `multiply_by_two'
 	# ./math_spec.rb:10:in `block (2 levels) in <top (required)>'  

Finished in 0.00227 seconds (files took 0.11411 seconds to load) 
3 examples, 1 failure  

Failed examples:  
rspec ./math_spec.rb:7 # .multiply_by_two will handle valid values

When an RSpec “it” block succeeds, we get a dot at the command-line: just enough information to know things are working. A successful run will be a string of these dots. However, instead of the three dots we expected, we got an F -- indicating a failed test -- and two dots. More helpful is that the framework also gives us details on:

  • What our expectation was (we should get 0)
  • What the actual result was (we received an ArgumentError)
  • What line of our application code caused the error (math.rb, line 3)
  • What line of the test framework found the error (math_spec.rb, line 9)

All of these clues should help us zoom in on the problem: I supplied the string “zero” as the argument when I should have supplied the number 0.

With that minor fix to the tests, we get a clean run:

> rspec math_spec.rb 
...  

Finished in 0.00315 seconds (files took 0.10478 seconds to load) 
3 examples, 0 failures

Now we get the string of dots that indicate success, along with some benchmark details. The file took 1/10 of a second to load and 3 milliseconds to run all our tests. I told you automated testing was fast!

Is that the same as TDD/BDD?

You may have heard about TDD (Test-Driven Development) or BDD (Behavior-Driven Development). They depend on automated testing to work, but are variations on the idea of “test-first” development, which means exactly what you think: writing the tests before you write the code that supplies the functionality.

At first glance, that might seem ridiculous: trying to test something that doesn’t exist yet? How do we even do that?

Let’s say you want to buy a bicycle to help you get around town. You could go to your local bike shop and ask for a “commuter bike”. They’ll sell you a bike that fits that general description, but on the ride home you realize it doesn’t fit your exact needs: the seat isn’t comfortable, the basket isn’t large enough for your groceries, the gears won’t get you up that last big hill.

Fortunately, bikes are built from components that the bike shop can swap in and out. Instead of going back to the shop and returning the whole bike, you can ask to try different seats, baskets and gears, exchanging different parts until you get exactly the bike you want.

But instead of starting with that not-quite-right bike and having to go all the way back to the shop to fix it, wouldn’t it be easier if you thought about all those requirements ahead of time? If you stopped to think about it, you’d remember that you need that big basket and that comfortable seat. Thinking over your requirements before you go shopping means you can do all your swapping on your first visit and never have to ride the not-quite-right bike out of the shop at all.

TDD lets you build your app the same way! First, you describe how it should behave using tests that expect that behavior. Then you write the code needed to make those tests pass. As the tests pass or fail, you can see how close your code is to doing what it should. And just like you can swap out components of that bike instead of returning the whole bike, you can swap out different pieces of your code to get it closer to doing what you want. Eventually, you get a solid line of dots and you know your code correctly handles every scenario you can think to throw at it. If you think of a new scenario, just add another test and then tweak your code until it succeeds as well, all with the “safety net” of your existing tests to make sure all your existing expectations continue to be met.

Speed + Communication + Confidence = Happiness

Automated tests run fast, communicate clearly with your future self and your team, and give you the confidence to experiment with your code without introducing bugs in a way that manual testing can’t.

Writing “good” tests that give maximum value is an entirely separate topic, and it does take time to think them up and figure out how to write them, but done well, their advantages clearly outweigh their problems.

So choose a framework and do some automated testing on your next project. I think you’ll find it an important milestone on the road to becoming the best developer you can be.