Note: This is a really simple example. But I’m quite confident that this could work on bigger problems / projects too.
On the train home from SoCraTes 2014, I was discussing with Johannes Link if REPL driven development could give us the same advantages we get from Test Driven Development (TDD).
We agreed that TDD gives us at least these three advantages:
- Fast Feedback: With a good unit test suite, you quickly get feedback about what is going on in your code base. Especially with a plugin like Infinitest
- Small, safe steps: By writing the tests first, you automatically validate every step. By writing good tests, you force yourself to take small steps.
- The tests: They are a living documentation of our thinking process and a regression suite for our code base.
That a REPL gives you fast feedback is obvious. But what about the other two?
Small, safe steps
We quickly agreed that the REPL allows you to take small, safe steps. You change a little bit of your code, then you immediately try it out. Let me show you with an example:
In this example I want to print all prime numbers up to a given number. For this I’ll implement a lazy sequence of prime numbers using the Sieve of Eratosthenes [1]. I’ll first solve the problem directly in the REPL, then I’ll copy the solution to a source file.
The sieve initially works on a list of all positive integers, starting at 2. So we’ll define this first.
(take 10 (iterate inc 2)) => (2 3 4 5 6 7 8 9 10 11) (defn to-sieve [] (iterate inc 2)) => (var user/to-sieve) (take 10 (to-sieve)) => (2 3 4 5 6 7 8 9 10 11)
Now, when we know a prime, we have to remove (“cross out”) all it’s multiples from the list of numbers. We’ll use “filter” for this.
(defn remove-multiples [prime s] (filter #(not= 0 (mod % prime)) s)) => (var user/remove-multiples) (take 10 (remove-multiples 2 (to-sieve))) => (3 5 7 9 11 13 15 17 19 21) (take 10 (remove-multiples 3 (to-sieve))) => (2 4 5 7 8 10 11 13 14 16)
We know that the first number in the list is a prime number. So we can define a lazy sequence “lazy-sieve” that takes a sequence of numbers as a parameter. Based on it we can define a function “sieve” that provides the list [2].:
(defn lazy-sieve [s]
(let [prime (first s)]
(cons prime
(lazy-seq
(lazy-sieve (remove-multiples prime (rest s)))))))
=> (var user/lazy-sieve)
(take 10 (lazy-sieve (to-sieve)))
=> (2 3 5 7 11 13 17 19 23 29)
(defn sieve []
(lazy-sieve (to-sieve)))
=> (var user/sieve)
(take 10 (sieve))
=> (2 3 5 7 11 13 17 19 23 29)
The lazy sieve removes the first number from the sequence of numbers - We know this must be a prime number. It combines this prime number with the lazy sequence of other prime number (“(cons prime (lazy-seq …))”). To create the lazy sequence, it simply removes all multiples of the current prime from the rest of the sequence and passes it to “lazy-sieve” again.
Now we can define “primes-up-to n”, which takes prime numbers from this lazy sequence as long as the current prime nubmer is less than n:
(defn primes-up-to [n] (take-while (partial > n) (sieve))) => (var user/primes-up-to) (primes-up-to 10) => (2 3 5 7)
Extracting the code
Now I can simply extract all functions I still need from the REPL to a clojure source file. This just copy/paste: Really simple.
primes.clj
(ns primes)
(defn to-sieve [] (iterate inc 2))
(defn remove-multiples [prime s]
(filter #(not= 0 (mod % prime)) s))
(defn lazy-sieve [s]
(let [prime (first s)]
(cons prime
(lazy-seq
(lazy-sieve (remove-multiples prime (rest s)))))))
(defn sieve []
(lazy-sieve (to-sieve)))
(defn primes-up-to [n]
(take-while (partial > n) (sieve)))
But what about the tests
On the train, Johannes and I were discussing how you could come up with tests after everything already works. I was like “Well, you could think about what exactly you want from a regression test suite and… Wait a minute, the tests are already there! You already have input/output pairs in the REPL that you validated as correct! Just generate the tests from them!”
Extracting automated tests
So now we can generate automated tests from the things we did in the REPL. Everytime I was trying a function with concrete parameters, the REPL told me the result, and I manually checked the result for correctness. So now I only have to find those results that were correct. I can use these to generate automated test cases. I only have to reorder the function call / result pair and wrap it in a call to “expect” and voila: I have a test.
So, this REPL output would result in the following test:
(take 10 (to-sieve))
=> (2 3 4 5 6 7 8 9 10 11)
(expect '(2 3 4 5 6 7 8 9 10 11)
(take 10 (to-sieve)))
This is a little bit more work than copy/paste, but not much more. Copy the function call and the result to a file, reorder, wrap them in “expect”. After extracting all the tests from the REPL, the resulting clojure file looks like this:
primes_test.clj
(ns primes-test
(:use [primes]
[expectations]))
(expect '(2 3 4 5 6 7 8 9 10 11)
(take 10 (to-sieve)))
(expect '(3 5 7 9 11 13 15 17 19 21)
(take 10
(remove-multiples 2 (to-sieve))))
(expect '(2 4 5 7 8 10 11 13 14 16)
(take 10
(remove-multiples 3 (to-sieve))))
(expect '(2 3 5 7 11 13 17 19 23 29)
(take 10 (lazy-sieve (to-sieve))))
(expect '(2 3 5 7 11 13 17 19 23 29)
(take 10 (sieve)))
(expect '(2 3 5 7)
(primes-up-to 10))
(run-all-tests)
Where running the last line results in the output:
Ran 6 tests containing 6 assertions in 14 msecs 0 failures, 0 errors.
Conclusion
Johannes then said “It would be nice if the IDE or the REPL could do this for you.” Indeed it would be great if you could generate test cases from a REPL input/output pair by just pressing a button. But as long as this functionality is not there, generating them by hand is not that hard.
For me this means that I can get all the advantages of TDD by using the REPL - At least with clojure. Maybe this might be a bit harder with other languages - I did not try it. Yet.
[1] Whether or not this is a good solution is another topic… [2] I know I could have combined the two. To me, separating them seemd clearer for now.
You might be also interested in:
- Simplicity: Simplify. Then simplify more. And: Don't let the code become complicated in the first place.
- Cheap plastic drills: Most people think construction workers should have great tools. A lot of people think paying more than 1000 Euros for an office chair is a waste of money, even for a software developer who uses it 8 hours a day. Good tools are expensive.
- Mocks or Intermediate Results: What I Would Do: An answer to Kent Beck's article, where he wrote about how he uses intermediate results instead of mocks. I show an alternative approach.
- Improve your Agile Practices: A FREE course that teaches you how you can improve as a software development team
