Archive for the ‘TDD’ Category
I like TDD. But if you reading this blog you probably knew that already.
“coding like a bastard” considered harmful
I can remember a time when I thought, because I’d been taught so and it seemed to make sense, that software—programs—were designed on paper, using ▭s and →s of various kinds and, in several of my earlier jobs, more than a few ∀s and ∃s and that the goodness of a design was determined by printing it out in a beautifully formatted document† and having an older, wiser, better—well, older and therefore presumably wiser, and more senior so presumably better, anyway—designer write comments and suggestions all over it in red pen during a series of grotesquely painful “review” meetings and then tying to fix it up until the older etc. designer was happy with it. After which came an activity know at the time as “coding like a bastard” after which came the agony of integration, after which came the dismaying emotional wasteland of the “test and debug” activity which took a duration essentially unbounded above, even in principle.
Hmm, now I come to write it down like that, it seems like that was a colossally idiotic way to proceed.
There were some guidelines about what made a good design. There were the design patterns. There were Parnas’s papers, such as On the Criteria… and there were all these textbook ideas about various kinds of coupling an cohesion and…ah, yes, the textbooks. Well, there was Pressman‘s, and there was Sommerville‘s and some more specialist volumes and some all–rans. When I returned to university after a fairly hair–raising time in my first job as a programmer, wanting to learn how to do this software thing properly, we used whatever edition of Somerville as current at the time—its now in its 9th—as our main textbook for the “software engineering” component: project management, planing, risk, that sort of thing.
So, it’s a bit…startling, we might say, to see Ian’s writeup of his experiment with TDD dealt with by Uncle Bob in quite such…robust, we might say, terms. Startling even for someone as…forthright, we might say, as I usually am myself.
As described, doing TDD wrongly
Thing is, though, Ian is, as described, doing TDD wrongly. And the disappointments that he reports with it are those commonly experienced by… by… by people who are very confident—rightly or wrongly and in Ian’s case, probably rightly—in their ability to design software well. I used to be very confident—perhaps wrongly, but I don’t think so—of my ability to design software. I mean to say, I could produce systems in C++ which worked at all—mid 1990s C++, at that—and this is no mean feat.
Interestingly, at the time I first heard about TDD by reading and then conversing with Kent and Ron and those guys on the wiki—C2, I mean, the wiki—I was already firmly convinced of the benefits of comprehensive automated unit testing, having been made to do that by a previous boss—who had himself learned it long before that—but of course we wrote the tests after we wrote the code, or, to be more honest about it, while debugging. And, yes, even with that experience behind me, I though that TDD sounded just crazy. Because it to someone used to the ▭s and →s and design as an activity that goes on away from a keyboard and especially to someone who does that well—or believes that they do—it does sound crazy.
And so a lot of the objections to TDD that Ian makes in his blog post seem early familiar to me. And not only because I’ve heard them often from others since I started embracing TDD.
Thorough, if unnecessarily harsh
Well, anyway, Bob’s critique of what Ian reports is pretty thorough, if unnecessarily harshly worded in places, but there are a few observations that I’d add.
Test-first or test-driven driven development (TDD) is an approach to software development where you write the tests before you write the program.
Apart from the fact that writing tests first is merely necessary, but very much not sufficient, for doing TDD, so far so good.
You write a program to pass the test, extend the test or add further tests and then extend the functionality of the program to pass these tests.
You build up a set of tests over time that you can run automatically every time you make program changes.
That does happen,yes…
The aim is to ensure that, at all times, the program is operational and passes the all tests.
Yep. I especially like the distinction between merely passing all the tests and also being operational.
You refactor your program periodically to improve its structure and make it easier to read and change.
and, sadly, this last sentence misses a key practice of TDD and largely invalidates what comes before. Which practice is that you refactor your code with maniacal determination up to as frequently as after every green bar.
Every. Green. Bar.
Technically, we could claim to be doing something “periodically” if we did it every 29th of February or every millisecond but I think that to say we do something “periodically” points to a lower frequency. But in TDD we should be refactoring often. Very often. Many times an hour. To really do TDD requires that we spend quite a large proportion of all the time invested any given programming exercise on refactoring. So, Ian has kind–of fallen at the first hurdle because he’s not really doing TDD right in the first instance.
Now, it used to be a frequent complaint about TDD advocates that we sounded like Communists: it was claimed that we would immediately respond to anyone who said that “I tried TDD and it doesn’t work” by claiming that they weren’t even doing TDD, really, in the same way that fans of Communism would contend that it had never really been tried properly so, hey, it might work, you don’t know.
Not a useful response
The thing is, though, a lot of people who dismiss TDD really haven’t tried it properly—and a lot who say that they do TDD aren’t doing it right either and are missing some benefits, but that’s another story—so of course they didn’t get the advertised effect. And by now we have lots of examples of people who really have tried TDD properly and the interesting and positive results they’ve obtained. Ian did not try doing TDD properly.
And then since that wasn’t going so well, he stopped even trying to:
[…] as I started implementing a GUI, the tests got harder to write and I didn’t think that the time spent on writing these tests was worthwhile.
Well, yes, we know that writing automated tests for GUIs is 1) hard and 2) relatively low value. But this:
So, I became less rigid (impure, perhaps) in my approach, so that I didn’t have automated tests for everything and sometimes implemented things before writing tests.
is not a useful response.
One useful response is to use something like MVC, or MVP, or ports-and-adaptors or one of the many other ways to make the GUI very, very thin and do automated tests behind that and test the actual GUI by hand. But from this point on Ian has basically invalidated his own exercise in TDD because although he wasn’t really doing it to begin with he was at least trying but it turned out to be tough and so he stopped trying. And also stopped learning. Which is a missed opportunity for him, and also for the rest of us. I encourage Ian to try again, maybe with some coaching, and see how that goes, because I would be genuinely interest to see how a seasoned software engineering academic gets on with that.
Not your daddy’s COBOL compiler
Think-first rather than test-first is the way to go.
he also says:
I started programming at a time where computer time was limited and you had to spend time looking at and thinking about the program as a whole.
Yes. There’s a whole hour long presentation that I have about this but—the microeconomics of programming have changed in quite a fundamental way over the last few decades. Even since I started working.
In my second job as a programmer I worked on a product written in C++ where, no joke, a full build was something you started on Friday lunchtime and went down the pub, hoping that it would be finished by the time you strolled in late on Monday morning. Even incremental builds on just the sub-system I was working on took “go have a cup of tea” amounts of time. Running our comprehensive automated unit test suite (written post hoc, as described above) took “go have lunch” amounts of time.
The time period that Ian is talking about was much worse even than that. In that era the rare and expensive resource was machine cycles and they need to be dedicated to doing the useful, revenue–earning thing. Programmer thinking time was, relatively, cheap and abundant so they mode of working tended to use lots of that to avoid wasting machine cycles on code that was not strongly expected to be correct.
If you wanted to work the way we do now—for example, with approximately one computer per programmer—you had to be, say, NASA, and you had to have, say, basically unlimited resources because your project was, say, considered to be a matter of national survival. But for most programmers, their employer could not afford that. The entire organisation might have as few as one computer. Maybe one to a department.
The whole edifice of traditional software engineering can be seen as a perfectly reasonable attempt to deal with the constraint that you can’t afford to have a programmer use machine cycles to do programming with. So you need to find ways to write programs away form a computer. That’s what the ▭s and →s were trying to do. The people who came up with that stuff meant well, but ended up creating that world of the colossally idiotic ways to proceed.
I was once sent on a COBOL programming course—it’s a long and dreary story—and on this course we worked within a simulation of those bad old good old days: programs were designed using what I later realised was Jackson Structured Programming, written out in pencil on pre-printed 80-column coding sheets, desk-checked, and then typed in to a COBOL development system. One PC for a class of about 20 students—before which we formed a queue—and we each only had three goes at the compiler. If it took more than three compile/test/debug episodes to get your program running you failed the course.
Today, we are awash with machine cycles. I have many billions of them available to me here right now every second and all I’m using them for is writing this blog post. John von Neumann* must be spinning in his grave.
Don’t play dumb
If I were programming right now, rather than doing this, then I could use those billions of cycles to get prompt, concrete feedback from a large body of tests and from other tools about my current position in a long series of small design decisions.
Rather than thinking in big, speculative lumps I could think in tiny, tiny increments—always with the ever important continual, frequent and determined refactoring.
There is a failure mode, though. Ian says:
[…] with TDD, you dive into the detail in different parts of the program and rarely step back and look at the big picture.
Don’t do that.
I don’t think that there’s anything in TDD that says not to step back and look at the big picture. There’s nothing that says to do that, it’s true, but why would’t you? It’s disappointing to see a retired Professor of Software Engineering playing dumb like this—if he feels the need to step back and look at the big picture then he should. He shouldn’t not do that merely because he’s making an attempt to try out a technique that doesn’t say to do that. I mean, really!
Mighty thinking is not the winning strategy
Added to which, I don’t recall anyone ever saying that TDD is the only design technique—and it is a design technique—that anyone needs to use at any scale to produce a good system. What is said, by me for one, is that by using TDD to guide design thinking and most importantly, to make it quick, easy, cheap and safe to explore different design options, we can get to better results sooner and more reliably than we can by mighty thinking, which was previously the only economically viable method.
I understand that this can discomforting to those who’s thoughts tend to the mighty. It’s almost as if in contemporary** software development mighty thinking has turned out not to be the winning strategy, long term.
Neither for individuals in their careers nor for their employers, nor for their industry. It might be time to come to terms with that. And for a certain kind of very smart, very capable, very confident designer of programs that means letting go. Letting go of the code, of the design, letting go of a certain sense of control and gaining in return a safe way to explore design options that you were too smart to think up yourself.
And that’s not easy.
† we had to use professional quality document preparation systems to do that, because of all the ▭s and →s and ∀s and ∃s. Which was fun.
* He’s supposed to have responded to a demo of some tools written by a programmer to make programming easier by saying that “it is a waste of a valuable scientific computing instrument to use it to do clerical work”
** that is, since about 2006…
Please add your comments about the session to this post.
Attendees, please add your thoughts, and links to your code repo if you wish, as comments to this post.
What’s the problem with TDD?
TDD is quite a simple process. Beck Describes it here in these terms:
- Add a little test
- Run all tests and fail
- Make a little change
- Run the tests and succeed
- Refactor to remove duplication
This turns out to be a hard thing to do. My observation has been that the more experienced and fluent a programmer someone is the more difficult it is for them to stick to this process. What tends to happen is more like this:
- Think of a solution
- Imagine a bunch of classes and functions that you just know you’ll need to implement (1)
- Write some tests that assert the existence of (2)
- Run all tests and fail
- Implement a bunch of stuff
- Run all tests and fail
- Run the tests and succeed
- Write a
TODOsaying to go back and refactor some stuff later.
Really good programmers can get away with this, for a bit. But even during that early period I think they are missing a trick. A couple of tricks, in fact.
Firstly, in the pseudo-TDD steps 1, 2 and 3 can take a long, long time. Tens of minutes perhaps, hours, or days even. This is time during which you aren’t running tests, aren’t getting feedback and aren’t learning anything. Step 7 must be assumed to take an amount of time unbounded above.
Secondly, in the Pseudo-TDD process the programmer must fall back on some technique for making design decisions and somehow getting them right exactly at the time when they know least about the problem and its solution. In TDD we have the advantages of evolutionary design: we can discover a good-enough design and then incrementally improve it. I think it is really hard for people who know themselves to be good programmers to let go of the design process in this way.
TDD as if you Meant It
I began to wonder if there was some sort of exercise that folks could do, in safe controlled conditions, whereby they could experience the odd and surprising (and delightful) things that can happen when you really do TDD, as Beck describes it, with the hope that the experience would carry over to their daily work as programmers.
This would be a pair-programming exercise, since it’s often easier to maintain a level of discipline if
you know someone is watching you can provide friendly, constructive feedback.
The problem to be solved should be simple enough that decent progress can be made on it during a typical conference “workshop” session (say, 90 minutes to 3 hours) and also have an obvious solution that experienced programmers would want to jump to implementing.
In the first couple of presentations of the session I used a problem from the game of Go. This worked reasonably well but I feel I spent too long explaining the game. Some folks who have picked the exercise up have used tic-tac-toe with good results. I tried that myself at NDC 2011 and was quite pleased with the result. From now on I will use that problem.
Another description of TDD is due to Bob Martin. It goes like this:
- You are not allowed to write any production code unless it is to make a failing unit test pass.
- You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
- You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
I took these rules as a starting point and then tried to produce stronger rules that would force the programmer (pair) to allow the design to evolve. I don’t think I’ve yet landed on the best set of rules, and people report difficulties with various part of it, but if I were going to do the workshop today these are the rules I would enforce (subject to change and refinement at any time, last updated 3 Sept 2011):
- Write exactly one new test, the smallest test you can that seems to point in the direction of a solution
- See it fail
- Make the test from (1) pass by writing the least implementation code you can in the test method.
- Refactor to remove duplication, and otherwise as required to improve the design. Be strict about using these moves:
- you want a new method—wait until refactoring time, then… create new (non-test) methods by doing one of these, and in no other way:
- preferred: do Extract Method on implementation code created as per (3) to create a new method in the test class, or
- if you must: move implementation code as per (3) into an existing implementation method
- you want a new class—wait until refactoring time, then… create non-test classes to provide a destination for a Move Method and for no other reason
- populate implementation classes with methods by doing Move Method, and no other way
The member of the pair without their hands on the keyboard must be very strict in enforcing these rules, especially 4.1 and 4.2
After some respectable time coding, contrast and compare solutions. Consider the classes created. How many? How big? What mutable state? Consider the methods created How many? How long? Apply a few simple design metrics. How was the experience of working this way different from the usual? How could these ideas be applied in your day job?
Experiences with the Workshop
If you haven’t tried the workshop yet, and would like to, you might want to stop reading now so that you don’t lose the “a-ha!”. That is to say: spoiler alert!
This is a tough exercise for experienced programmers and doubly so experienced TDD practitioners. I observe that pairs including folks who are not full-time programmers (BA’s, testers, managers even) do much better.
I’ve come to recognise the point about 5 to 10 minutes after the start of the exercise proper where everyone quietens down and seems to be making progress. At this point I stop the exercise and ask who has (in the case of tic-tac-toe, say) created a class called something like
Board with something like a 3×3 array of
ints in it (or even better, of an
enum with members like
O) and no tests for it. After a bit of cajoling it always turns out that several pairs have. Because they “know” they will “need” it. Or because without that class they “can’t write any tests”.
At this point the facilitator needs to be strong and force them to delete that code and start again. Yes, really.
It’s extraordinarily hard for some pairs to get going. Often it’s the ones who have just had their code deleted. They will just sit and stare at an empty editor window. This is the crucial learning step. If they are not allowed to write tests about the solution that doesn’t exist yet, what are they allowed to write tests about? There is only the problem. Here is where we start to see the connection between TDD, BDD (in so far as they are not identical—hint: they are identical) and DDD and eDSLs.
There is always a startling variety of solutions.
Some pairs can implement pretty much a whole tic-tac-toe game playing program without a class remotely like a
At least these people have kindly written up their experiences of doing the exercise:
- The first outing for the exercise was at Software Craftsmanship 2009 and Gojko Adzick did an excellent writeup of that and then tried running the exercise himself
- Rob Bowley has done the exercise a couple of times and shares his thoughts on it
- Mark Needham used the exercise (with rock-paper-scissors as the target) in a code Dojo over two weeks, with various observation in week 1 and week 2 and as a more extended exercise himself
Those are just the ones I know about. I’d love to hear about more.