TDD as if you Meant It
What’s the problem with TDD?
TDD is quite a simple process. Beck Describes it here in these terms:
TDD
- Add a little test
- Run all tests and fail
- Make a little change
- Run the tests and succeed
- Refactor to remove duplication
This turns out to be a hard thing to do. My observation has been that the more experienced and fluent a programmer someone is the more difficult it is for them to stick to this process. What tends to happen is more like this:
Pseudo-TDD
- Think of a solution
- Imagine a bunch of classes and functions that you just know you’ll need to implement (1)
- Write some tests that assert the existence of (2)
- Run all tests and fail
- Implement a bunch of stuff
- Run all tests and fail
- Debug
- Run the tests and succeed
- Write a
TODOsaying to go back and refactor some stuff later.
Really good programmers can get away with this, for a bit. But even during that early period I think they are missing a trick. A couple of tricks, in fact.
Firstly, in the pseudo-TDD steps 1, 2 and 3 can take a long, long time. Tens of minutes perhaps, hours, or days even. This is time during which you aren’t running tests, aren’t getting feedback and aren’t learning anything. Step 7 must be assumed to take an amount of time unbounded above.
Secondly, in the Pseudo-TDD process the programmer must fall back on some technique for making design decisions and somehow getting them right exactly at the time when they know least about the problem and its solution. In TDD we have the advantages of evolutionary design: we can discover a good-enough design and then incrementally improve it. I think it is really hard for people who know themselves to be good programmers to let go of the design process in this way.
TDD as if you Meant It
I began to wonder if there was some sort of exercise that folks could do, in safe controlled conditions, whereby they could experience the odd and surprising (and delightful) things that can happen when you really do TDD, as Beck describes it, with the hope that the experience would carry over to their daily work as programmers.
This would be a pair-programming exercise, since it’s often easier to maintain a level of discipline if you know someone is watching you can provide friendly, constructive feedback.
The problem to be solved should be simple enough that decent progress can be made on it during a typical conference “workshop” session (say, 90 minutes to 3 hours) and also have an obvious solution that experienced programmers would want to jump to implementing.
In the first couple of presentations of the session I used a problem from the game of Go. This worked reasonably well but I feel I spent too long explaining the game. Some folks who have picked the exercise up have used tic-tac-toe with good results. I tried that myself at NDC 2011 and was quite pleased with the result. From now on I will use that problem.
Another description of TDD is due to Bob Martin. It goes like this:
- You are not allowed to write any production code unless it is to make a failing unit test pass.
- You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
- You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
I took these rules as a starting point and then tried to produce stronger rules that would force the programmer (pair) to allow the design to evolve. I don’t think I’ve yet landed on the best set of rules, and people report difficulties with various part of it, but if I were going to do the workshop today these are the rules I would enforce (subject to change and refinement at any time, last updated 3 Sept 2011):
The Rules
- Write exactly one new test, the smallest test you can that seems to point in the direction of a solution
- See it fail
- Make the test from (1) pass by writing the least implementation code you can in the test method.
- Refactor to remove duplication, and otherwise as required to improve the design. Be strict about using these moves:
- you want a new method—wait until refactoring time, then… create new (non-test) methods by doing one of these, and in no other way:
- preferred: do Extract Method on implementation code created as per (3) to create a new method in the test class, or
- if you must: move implementation code as per (3) into an existing implementation method
- you want a new class—wait until refactoring time, then… create non-test classes to provide a destination for a Move Method and for no other reason
- populate implementation classes with methods by doing Move Method, and no other way
The member of the pair without their hands on the keyboard must be very strict in enforcing these rules, especially 4.1 and 4.2
After some respectable time coding, contrast and compare solutions. Consider the classes created. How many? How big? What mutable state? Consider the methods created How many? How long? Apply a few simple design metrics. How was the experience of working this way different from the usual? How could these ideas be applied in your day job?
Experiences with the Workshop
If you haven’t tried the workshop yet, and would like to, you might want to stop reading now so that you don’t lose the “a-ha!”. That is to say: spoiler alert!
Mine
This is a tough exercise for experienced programmers and doubly so experienced TDD practitioners. I observe that pairs including folks who are not full-time programmers (BA’s, testers, managers even) do much better.
I’ve come to recognise the point about 5 to 10 minutes after the start of the exercise proper where everyone quietens down and seems to be making progress. At this point I stop the exercise and ask who has (in the case of tic-tac-toe, say) created a class called something like Board with something like a 3×3 array of ints in it (or even better, of an enum with members like BLANK and X and O) and no tests for it. After a bit of cajoling it always turns out that several pairs have. Because they “know” they will “need” it. Or because without that class they “can’t write any tests”.
At this point the facilitator needs to be strong and force them to delete that code and start again. Yes, really.
It’s extraordinarily hard for some pairs to get going. Often it’s the ones who have just had their code deleted. They will just sit and stare at an empty editor window. This is the crucial learning step. If they are not allowed to write tests about the solution that doesn’t exist yet, what are they allowed to write tests about? There is only the problem. Here is where we start to see the connection between TDD, BDD (in so far as they are not identical—hint: they are identical) and DDD and eDSLs.
There is always a startling variety of solutions.
Some pairs can implement pretty much a whole tic-tac-toe game playing program without a class remotely like a Board.
Others
At least these people have kindly written up their experiences of doing the exercise:
- The first outing for the exercise was at Software Craftsmanship 2009 and Gojko Adzick did an excellent writeup of that and then tried running the exercise himself
- Rob Bowley has done the exercise a couple of times and shares his thoughts on it
- Mark Needham used the exercise (with rock-paper-scissors as the target) in a code Dojo over two weeks, with various observation in week 1 and week 2 and as a more extended exercise himself
Those are just the ones I know about. I’d love to hear about more.
cumulativehypotheses 
[…] TDD-coach. Under dagen presenterade Emily olika utmaningar för varje iteration. Till exempel TDD as if you meant it (också beskrivet här). Det var en mängd olika språk som användes under dagen: JavaScript, […]
Valtech Labs » Code Retreat på Valtech
September 21, 2011 at 11:20 am
I think there’s a typo. Shouldn’t step 1 be “Write a little test”?
Nat
October 6, 2011 at 9:42 pm
Actually it’s “Add a little test”. Fixed.
keithb
October 7, 2011 at 9:14 am
[…] TDD as you meant it […]
The Global Day of Coderetreat in Cologne « Kurt's Blog
December 5, 2011 at 4:49 pm
I’d like to get a better understanding for how this approach would work in a UI environment given the mainstream TDD tooling available today, such as Selenium. Additionally, what advice can given to organizations where a significant amount of time (minutes) are required to switch contexts from testing to development?
Josh Habdas
January 4, 2012 at 7:38 pm
Hi Josh,
I’m not sure that the exercise would work very well with a GUI. Attendees at the sessions do a command-line interface (if any). I do say during the sessions that I don’t necessarily recommend this way of working for anyone’s day job.
My advice to the kind of organization that you mention is that they should change their tools and practices so that there is no context switch between development and test. That’s independent of whether or not they do TDD int he sense of the exercise described above. What sort of context switch to do you have in mind?
keithb
January 6, 2012 at 10:47 am
If backed by mock data applications may need to be built in a “mock data mode” to support the development or execution of automated tests, causing an increase in development overhead. Some systems may not be capable of or conducive to development activities while running backed by mock data, causing yet more overhead as developers shift back-and-forth between test and actual development – the context shift I’m referring to.
Josh Habdas
January 14, 2012 at 7:35 pm
Hi Keith,
I’m really interested to give this exercise a go. Our approach at the moment seems quite pseudo-TDD, and the bits and pieces which I’ve read, or that people have told, have pointed towards TDD being most valuable as a tool for design, rather than to verify a design created independently of tests.
However, I’m concerned by your comment of not necessarily recommending this way of working in our day jobs. What did you mean by that? Is it worth me learning a process which I’m not going to use in my day job? I just worry that the comment is going to give my boss the ammunition to say that it’s fine to do a code kata or an exercise using the above rules, but we’ll do it differently when we write production code.
Douglas Waugh
October 1, 2012 at 11:22 am
Hi Douglas,
the comment about day jobs really applies to the strict rules of the exercise, the part about only writing new code inside a test method and so on. I don’t necessarily recommend going that far in production coding (although sometimes I do personally go that far). However, I absolutely and very strongly recommend using TDD in production coding as a design tool. And if your boss is in central London I’ll happily come and say so 🙂
keithb
October 1, 2012 at 4:52 pm
I had a brief play with technique and had one question arise from my experience. When writing test code, i often want to express the fact that there is input data or data in state (such as a DB) that i know has to impact the outcome of the test. I write this and then it often ends up unused. Should refactoring remove this? Or is the implementation not valid?
Dafydd
February 20, 2012 at 9:10 am
Reblogged this on weblabs42.
carlozamagni
December 17, 2012 at 7:03 pm
[…] https://cumulative-hypotheses.org/2011/08/30/tdd-as-if-you-meant-it/ […]
Linkpool Januar 2013 |
January 22, 2013 at 6:30 pm
[…] code at different levels of abstraction. ** – Think about TDD in terms of states and moves. – do TDD as if you meant it – concentrate on refactoring in very small […]
Code Retreat Stockholm | Coding Is Like Cooking
May 8, 2013 at 9:40 am
[…] isn’t time for anything else. This chimes with what Keith Braithwaite is getting at in his TDD as if you meant it, and gives the lowest barrier to getting the first test passing. This is also a good illustration […]
Growing Legacy Code, Guided by Tests | ianhaworth
August 4, 2013 at 9:05 pm
[…] come from TDD as if you meant it. They force micro steps and really slow you down to make you think about what you can test […]
Where to start | Accept no broken windows
October 5, 2013 at 1:19 pm
[…] like to move the work to actual implementations of the application. It is similar to the concept of TDD like you mean it, but I don’t think I strictly adhere to that principle. In any event, we can move the […]
The Making of a Test-Driven Grocery List Application in JS: Part II | InfoLogs
October 5, 2013 at 9:00 pm
[…] JavaScript Development by Christian Johansen Introducing BDD by Dan North TDD as if you Meant it by Keith Braithwaite RequireJS AMD Jasmine Sinon […]
The Making of a Test-Driven Grocery List Application: Part VIII | InfoLogs
October 5, 2013 at 9:05 pm
[…] JavaScript Development by Christian Johansen Introducing BDD by Dan North TDD as if you Meant it by Keith Braithwaite RequireJS AMD Jasmine Sinon […]
The Making of a Test-Driven Grocery List Application in JS: Part IX | InfoLogs
October 5, 2013 at 9:06 pm
[…] JavaScript Development by Christian Johansen Introducing BDD by Dan North TDD as if you Meant it by Keith Braithwaite RequireJS AMD Jasmine Sinon […]
The Making of a Test-Driven Grocery List Application in JS: Part X | InfoLogs
October 5, 2013 at 9:07 pm
[…] Think about that for a minute. The choice of exploration vs. specification captures something important and beautiful about the nature of software development. The discovery, learning, process of understanding… This aspect of software development is discussed in topics such as Emergent Design, and TDD. […]
Sometimes it’s about attitude, or “You don’t really know the future” | Dan Bernardic
December 12, 2013 at 5:30 am
[…] which is not actually TDD from what I understand based on the clarifications in places like TDD as if you meant it . In that article Keith Braithwaite calls the approach IRCMaxell describes as […]
Quasi-Editorial corrections of DIRTI by @ircmaxell | Dan Bernardic
January 7, 2014 at 8:56 pm
I have tried doing this exercise _VERY_ strictly, but I am a bit stuck at what you mean by “Be strict about using these moves”. Do you mean that Extract Method and Move Method are the only refactorings you can do? If so, then you only end up with essentially static methods in classes without state, meaning all state is passed in as parameters.
This means a method such as takeField() must have a lot of parameters, including previous player and current player, for it to ensure that players take turns.
Having the possibility of internal state in the classes would make this quite unnecessary, but I am not certain how to get there if this is not on the list of permitted “moves”. Could you clear this up?
Carl-Erik
February 18, 2014 at 5:02 pm
That is a bit ambiguous, yes. what I mean is, be strict about only creating a class as the target of a move method, and only create methods by extracting them from tests. You certainly can use other refactorings!
Then again, a functional programmer might tell you that this:
is a good thing 🙂
keithb
February 18, 2014 at 6:15 pm
[…] emerge. After a long discussion, Janasked me if I could do again a coding demonstration on TDD as if you meant it on the example we discussed: tic-tac-toe. So I committed to add a session in the afternoon on […]
Sharing and Learning in the European Communities: SoCraMOB : Mozaic Works
March 16, 2014 at 9:24 pm
[…] TDD as if you Meant it – Cumulative Hypotheses […]
Slide Deck for So Long and Thanks for All the Tests | Scottish Developers
May 2, 2014 at 12:13 pm
[…] even had a session where we combined TDD as if you meant it with the extract pure function refactoring. This was an interesting exercise but we just ended up […]
Legacy code retreat | jvaneyck
July 27, 2015 at 11:01 am
[…] like the idea of Tdd as if you meant it and this was the result from my implementation. Your job, if you want to use this solution, is to […]
Packaging a zip file from Java using Apache Commons compress | Thomas Sundberg
August 21, 2015 at 2:10 pm
[…] a design, that is, we shouldn’t come up with a design and then test for it. Instead we discover a design through writing tests. TDD doesn’t design for you, but it does give you a set of behaviours within which to do […]
Mocks | cumulativehypotheses
November 3, 2015 at 9:43 pm
[…] this exercise: TDD as if you meant it (Another […]
Unit test adoption | ASK AND ANSWER
January 8, 2016 at 1:25 am
[…] a book, but a blog post (TDD as if you meant […]
Books from Codemotion Milan 2016 – Random code & thoughts
November 27, 2016 at 6:32 pm