[Code Friday] Tasty, tasty behaviour-driven development

[maradydd]

By way of sfllaw, a development paradigm I had not previously known about, and tool for developing in this fashion. Holy shmoley. I agree with Bill Tozier, I want this for Python yesterday.

Behaviour-driven development is basically test-driven development on steroids: it takes the principle we like to cite, "write your man pages first!", and hooks it right into the test-driven development cycle, except now you're developing one behaviour at a time, so you can write your tests piece by piece and have individual chunks of the system piece by piece. I like TDD, but sometimes I have to write code fast (and yes, TDD always ends up saving me time in the end, but we've all had those projects where OMG EVERYTHING IS ON FIRE AND THERE'S NOT TIME TO DO IT RIGHT. Behaviour-driven development eliminates your excuses to not do it right: you're producing code as discrete functional units, complete with tests to prove that they are correctly functioning functional units, and you're producing it fast enough to keep management/the client happy. (Clients are sometimes not happy when the first week of work goes into building the unit test suite. Yes, yes, I know, that week of work saves a month or more later on down the line. Some of my clients are no longer my clients for a reason.)

Behaviour-driven development is also a great tool for the "design the UI first" school of programming, and any project that doesn't follow that school of programming is doing it wrong. (Think of it this way: if you're writing a library, design the API first -- that is to say, write the man page first. If you're writing a web application, mock up the user interface, figure out what the damn thing's going to look like and do all your changing-your-mind about how the UI is going to behave before you start laying down AJAX requests.)

Also courtesy sfllaw, a talk by Ben Mabey explaining not only these ideas but the business decisions which motivate behaviour-driven development. This is a really great overview and I strongly encourage any programmer with a pragmatic spirit -- or, even better, an entrepreneurial one -- to block out half an hour of your time to watch it.

Alas and alack, Cucumber is not available for Python yet, and from what I've seen, I really like the way it works. It apparently can be used with PHP, but I really would prefer to avoid PHP if at all possible; my preferred style is just way too functional these days to blend well with PHP. (I've developed a thing for continuation-passing style in the last month or so.) This may end up being the thing that finally motivates me to learn Ruby. I have a little side project going on right now that has a web-application-framework-shaped hole in it, and I had been planning on using Django, but given that it's going to be a Javascript-heavy front end with likely a healthy dose of script.aculo.us, Rails could be a better tool for the job. I'll need to decide if I like how Rails talks to databases; I'm madly in love with the way Django does it and anything less will be a major disappointment, so this is definitely a factor to consider. (Current Rails devs, your input is welcome -- I know very little about your framework. I used to be cranky about the lack of integration with Apache, but there's mod_rails these days and I assume that removes a lot of the reasons I had for bitching.)

And I'd have real continuations. That's always a plus.

Decisions, decisions. But I do like the fact that tools like this exist at all; it's me who needs to get over my uncanny-valley problem with Ruby.

(karnythia, thewayoftheid, tanyad, I'm not talking about the project I'm doing for y'all, this is a different project. So many irons in the fire!)

Comments

[sfllaw.livejournal.com]

I am very happy with the way Django makes RESTful APIs easy. As such, it is JavaScript-friendly, whether you use a library or not.

We use JQuery and have lots of success.

[maradydd.livejournal.com]

Oh, yes. I particularly mention Rails and script.aculo.us because, rik has informed me, the two are integrated about as well as Django and $your_database are. I'm down with programmatic control over dynamically generated user interface elements (which this project will involve). When the user is creating state that the server should have the ability to know about, that's a decent enough reason.

[sfllaw.livejournal.com]

Rails and script.aculo.us are integrated quite well, far better than what Django gives you.

But then again, I suppose it's a differing philosophy thing: http://lethain.com/entry/2008/sep/21/intro-to-unintrusive-javascript-with-django/

[maradydd.livejournal.com]

Hurr. This particular application may be so rich-GUI that it might not be possible to implement a Javascript-free version, much as good standards adherence shakes its fist at me. (I'll be happy to discuss it over IM if you want to hear more, or perhaps help me brainstorm ways to be nice to them what hates Javascript.) But the article is thought-provoking, and I will meditate on it as I turn our mockup into CSS/HTML.

[digitalsidhe.livejournal.com]

FWIW: Rails also makes RESTful APIs very easy. (And Prototype goes along with that, in the way it handles AJAX request returns.)

While Rails is very well integrated with Rails, they're not very tightly integrated. I was worried about that, but it turns out it's pretty easy to use other JavaScript frameworks with Rails; Rails doesn't give you any crap if you just drop in MooTools or jQuery instead.

(Plus, of course, jQuery does a bang-up job of cooperating with other JS frameworks. I was pretty impressed when I saw how easy it is to use jQuery alongside anything else.)

[maradydd.livejournal.com]

Oh, hey, that's good to know, since MooTools might actually be a better hammer for this job. (I am saying this based on having developed a UI in script.aculo.us, and having played with UIs that other people have developed in MooTools; I will have to have a look at the docs.)

[sfllaw.livejournal.com]

Oh, also, don’t you have real continuations now that generator.send() works?

http://docs.python.org/reference/expressions.html#yield-expressions

[maradydd.livejournal.com]

My understanding was that you didn't get real continuations until you also got coroutines, but I could be wrong about that. (They seem to be working on this, though.)

[sfllaw.livejournal.com]

You have coroutines with generators as of Python 2.5. Rejoice!

[maradydd.livejournal.com]

Oh cool, it wasn't clear to me whether PEP 342 had been implemented or not. *rejoices*

[kragen.livejournal.com]

No. Here are some things you can do with real continuation support that you can't do with generators in Python:

• Pass a continuation to a subroutine, which then invokes it. (You can pass a generator to a subroutine, but the generator can't pass its ability to yield a value to its caller to a subroutine.)
  
• Return from the same function call more than once.
  
• Unwind the stack when, say, you have an error or a recursive search finds what it was looking for.
  

The surprising thing is that it turns out to be possible to implement such a general construct efficiently.

[mutiny.livejournal.com]

What's wrong with using Django? It does the javascript/ajaxy/fuckery just as well as Rails, and it does everything else better as well.

[maradydd.livejournal.com]

Yes but can Django do this (http://maradydd.livejournal.com/359041.html?thread=1654913#t1654913)?

(pardon the grey-on-white code on that, rik wrote that comment before I changed my journal's theme)

If it can, well, then thanks for schooling me. :) I expect this interface to have dynamically generated Draggables and Sortables, so a programmatic interface to them will be a thing of Awesome.

[mutiny.livejournal.com]

Why wouldn't you just put that stuff in your template?

[maradydd.livejournal.com]

Yeah, but it's less crap to have to put in the template, you dig? Same way that using models means never having to write direct SQL queries.

[mutiny.livejournal.com]

I'm not seeing an appreciable difference between just using Effect.Highlight except you're adding an extra level of unnecessary indirection.

[mutiny.livejournal.com]

And ofcourse somebody made a link_to_remote clone for Django anyways.

http://www.djangosnippets.org/snippets/929/

[maradydd.livejournal.com]

Oh, hey, that looks handy. Thanks for the link!

[sfllaw.livejournal.com]

Oh yuck! They’re totally breaking everything by putting nonsense into your templates. Templates that render HTML should look like HTML, not PHP.

[mutiny.livejournal.com]

wut?

[sfllaw.livejournal.com]

The thing I like about Django templates is that they look like HTML.

There are a lot of things wrong with them, like the {% if %} tag, but looking and behaving like HTML is a very good thing.

Also, I find it super trivial to implement extensions to the template language, which means that I can push complexity out of the templates and into tags, filters, forms, fields, special helper objects, or even the models themselves; depending on where this belongs.

[maradydd.livejournal.com]

Also, I find it super trivial to implement extensions to the template language

...I am ashamed to admit that I never even thought of that.

[sfllaw.livejournal.com]

In Django, we use various views that spit out partial results, which you would have to write anyway, and use jQuery for the loading and the effects.

For rik's example of:

<%= link_to_remote "Submit to AJAX",
    :update => "dom_id_that_to_be_updated", 
    :url => { :action => "reload" },
    :complete => visual_effect(:highlight,
        "posts", :duration => 0.5) %>

we'd write, by hand, using jQuery:

<a href="#"
   onclick="$('posts').load('{% url posts.views.partial %}', {}, 
                            function() {
                                $(this).effect('highlight', {}, 3000)
                            })">
    Submit to AJAX
</a>

Granted, that’s a lot more code, but there’s a lot less magic. You can, of course, implement a lot of this magic in template tags. Simple Googling shows that this has already been done, but rather poorly from my point of view.

The Django way should probably look like this:

<a href="#" onclick="{{ posts.ajax.reload|effect:highlight }}">
    Submit to AJAX
</a>

using context variables with proper methods to render the JavaScript and some handy filters to add any effects.

[maradydd.livejournal.com]

I'd be all about that one-liner approach. It's ridiculously readable, and one of my favourite parts of Python is its high readability even when written in a very compact style.

[mutiny.livejournal.com]

I agree with this.

Also I think there might be generic views that output JSON anyways so you might not even have to go through the trouble of writing the views yourself.

[sfllaw.livejournal.com]

You mean django.core.serializers.serialize("json", SomeModel.objects.all())?

[mutiny.livejournal.com]

No I mean these: http://docs.djangoproject.com/en/dev/ref/generic-views/

Now that I look there isn't one, but that's pretty close to generic as you can get. Writing a generic view to just wrap around a Model and return JSON would be super easy.

[neoliminal.livejournal.com]

Some days I wish I wasn't a Drupal guru.

Sigh. I miss Python.

[digitalsidhe.livejournal.com]

I was going to say "Imagine I just quoted your entire third paragraph here", but this comment is going to be so far down the page that making readers scroll back up, count, and then scroll back down would just be mean, so I will quote the whole damn thing:

Behaviour-driven development is also a great tool for the "design the UI first" school of programming, and any project that doesn't follow that school of programming is doing it wrong. (Think of it this way: if you're writing a library, design the API first -- that is to say, write the man page first. If you're writing a web application, mock up the user interface, figure out what the damn thing's going to look like and do all your changing-your-mind about how the UI is going to behave before you start laying down AJAX requests.)

This is very similar to some stuff I wrote a long time ago — particularly the last two paragraphs of the "Interface and Program Better Adapted to Users" subsection.

Needless to say, I completely agree with you. :)

Agile's whole attitude toward documentation (namely, that it's something optional) is one of the things that's always annoyed me about it. I think BDD is a little lightweight, as docs go, but it's damned well better than nothing.

As someone who's been flipping back and forth between PHP, Ruby/Rails, and JavaScript for the past few months: If your style's been going more and more functional, I agree that you should avoid PHP. It has a few sops toward functional programming, but they're pretty weak sauce. In fact, they're downright laughable. Perl's FP options are much better.

Ruby, on the other hand, rather likes FP. In fact, the way I started learning FP was in JavaScript, using Prototype and Scriptaculous... which in turn got all their FP-ness from Ruby and Rails. Ruby likes FP a lot. (Okay, not as much as it likes OO-ness... and one thing it inherited from Perl is that it doesn't enforce either of those things on you. If you want to write procedural Ruby, you actually can... you'll just be wasting most of the language. It'd be like driving a car and never getting past second gear.)

As for Rails and Apache: Phusion Passenger makes it dead easy. If Apache integration was your major stumbling block to Rails adoption, then OMG you have got to check that thing out. Passenger is mod_rails in the sense that it's as easy and tight as mod_perl or mod_php. (I think some other project has the actual "mod_rails" name, but Passenger leaves it in the dust.)

If you like Python, I suspect you will either like Ruby a lot, because there are so many similarities, or hate it, because it's so close and yet so different in so many details. But they occupy very mich the same ecological niche (and Perl used to occupy pretty much the same one, before it faded into the background).

[maradydd.livejournal.com]

The problem I keep running into with Ruby is that it's so Python-like that I run into this uncanny-valley problem with the idioms. This is, however, a failure on my part with respect to Ruby.

There are a bunch of considerations to take into account when deciding what fits best in that web-application-framework-shaped hole: what the team members are most experienced with ($collaborator knows no Python yet, but might know some Ruby; I know very little Ruby and buggerall about Rails, but there are tutorials), what will be the best tool for the job with respect to features, what will be the best tool for the job with respect to maintenance and the Next Jackass Problem, just how badly I want to use this BDD tool, &c.

[digitalsidhe.livejournal.com]

The problem I keep running into with Ruby is that it's so Python-like that I run into this uncanny-valley problem with the idioms.

I can totally understand that. It's part of what I was getting at in my last paragraph. "Uncanny valley" is actually a pretty good phrase to describe how similar the two are (and yet how different).

just how badly I want to use this BDD tool, &c.

Which one? Cucumber? It's available for Rails. (I'm misunderstanding, aren't I?)

(Oh, and as far as Ruby tutorials? If you're as much like me as I think you are, you'll find why's Poignant Guide to Ruby to be interesting, but quickly infuriating and not very useful. It's fun to read the first 2 or 3 chapters, but by the end of them, I was going, "That's amusing as all hell, and I love the cartoon foxes screaming about chunky bacon,, but it's not actually teaching me to program in this language nearly as quickly as a more left-brained guide would.")

[maradydd.livejournal.com]

(I'm misunderstanding, aren't I?)

Yes. :) "Compatible with Cucumber" is a plus for Ruby and a minus for Python. Though, looking further into the Cucumber docs, it looks like there's a way to use it with some other tools for language-agnostic, outside-in Web application development; I may end up trying this, and if I do, I'll report back what I find out.

Thanks for the tip on the tutorial, I'll have a look. :)

[kragen.livejournal.com]

Perl's FP options are much better, yes, but that's because it supports FP better than any other mainstream programming language (by which I mean C, C++, C#, Java, Perl, Python, PHP, Ruby, JavaScript, and Visual Basic). Even Ruby is still kind of second-class in my book.

[ab3nd.livejournal.com]

"if you're writing a library, design the API first"

Yes, this. For some reason, it didn't occur to me that there is another way to do this until you pointed out that it's a good plan.

[maradydd.livejournal.com]

It is one of those things that needs to be taught, but all too often is just taken for granted. I wish schools and programming books both spelled this out in glaring detail.

[kragen.livejournal.com]

Sometimes doing some stuff bottom-up is a good idea, too; APIs need to be both easy to use and efficiently implementable.

[graydon.livejournal.com]

I'm not sure. The TDD stuff always seemed pretty cultish to me. Maybe I just work on weird programs. I work on two kinds of system.

System-type #1 is one where we're exploring a technical idea, and we don't know what's going to work or how or anything. So it's pure implementation. There's no "interface". It's a technical experiment. We write tests, and those are all hypotheses. Not "requirements".

System-type #2 is a language or machine tool of which there's a well understood set of requirements. They're expressed in terms of a very detailed spec and a ginormous testsuite (tens of thousands of cases), every one of which is completely non-negotiable. These systems typically acquire secondary testsuites as they mature -- the regression suite, the performance suite, etc. -- but again, there tends to be no room for misunderstanding: regression tests are bugs, performance tests are scalar measurements.

In both type of system, the implementation is the meat of the problem; the "surface area" (while typically infinite) is not something we spend any time with customers or "system-consumers" arguing over. So I think the brow-beating about using one or another "TDD methodology" variant ... is directed at a different kind of software?

[maradydd.livejournal.com]

So, for me, TDD is about minimizing my own fallibility. Many of the functions for which I write tests are never exposed in an API. (Yes, really.) This helps me to pinpoint places where I've screwed up, by showing me places where I haven't screwed up; if something bar() does is based on the result of foo(), and I can demonstrate to my own satisfaction that foo() is behaving properly, then if bar() is misbehaving then that's one less thing I have to check. Assuming I wrote my tests correctly.

I know where I tend to get lazy in areas that may end up biting me later, and I think this approach will help lessen my laziness in such a way that I'm more productive with the lines of code I write. YMMV.

Note that most of what I write are either scientific libraries or user-facing applications.

[graydon.livejournal.com]

Oh yeah, I'm quite paranoid about my own fallibility too. My code tends to be ~1/3 asserts, unless it's in some hyper-static language that eliminates possible failure modes by case decomposition or something.

Do massive quantities of asserts count as TDD? :)

[kragen.livejournal.com]

DBC and TDD/BDD are clearly closely related ways of thinking, but I think they have slightly different advantages and disadvantages.

They both tell you where the problem is when something goes wrong (but DBC is better at that).

They both provide documentation about how to call things (but BDD is better at that).

They both help you think out the requirements for what the code you're writing has to do (but BDD is better at that).

They both make your software more robust (but DBC is better at that, because it checks real values instead of fake ones).

They both reduce the effort of refactoring your software (but BDD is better at that).

They both reduce the time from when you write a bug to when you find it (although arguably BDD is better at that, because you don't have to test anything manually in order to make things blow up).

Unfortunately my experience with DBC (even in the informal sense you describe) is minimal. Anyone with experience with both who can confirm or deny?

[barbarienne.livejournal.com]

Speaking form the position of I-understand-about-one-word-in-three ignorance, I always assumed that programmers started with the UI and then worked out from there.

I mean, that seems so obviously sensible. Why would anyone do it the other way? Do they not know any people?

And then it occurred to me that this explains Windows perfectly. Light bulb!

[maradydd.livejournal.com]

I always assumed that programmers started with the UI and then worked out from there.

The world would be a better place if this were the case, but alas, no; far too much software is written from the perspective of "Algorithm Foo provides an excellent solution to Problem Bar; let's implement it! Oh, I guess we'll have to slap a UI on it to turn it into a tool..."

I am being rather generic in my use of the term "user interface"; by this I mean "whatever the user of the software actually has to touch." For a web browser, a music player, an email application, &c, this is the GUI. For a library, it's the API ("application programming interface", or set of functions which the library makes available for you to use). For a command-line tool, like 'ftp' or 'cd' or 'wget' or whatever, it's the arguments and flags that the user includes when invoking the command, and any special magic about them (cf. my brainfart last week about the ordering of libraries in an 'ld' command). For a programming language, it's the syntax of the language itself.

Another reason for things being done in clunky ways is "well, it's what people are used to, we can't just go reinvent the wheel." For all that C is annoyingly difficult to read, for instance, it's the one programming language that every competent programmer is expected to know. As such, many languages are rather C-like in their syntax, because if you know C, then you can pick up $foo fairly quickly. Java may well be replacing C, given the way that schools teach their CS curricula these days, but apart from the object-orientation and the lack of pointers, Java owes a lot to C syntax-wise as well. It's like we just can't get rid of this legacy that we don't want.

Python port

[skippy-fluff.livejournal.com]

Have you/any of the commenters seen the Python port of cucumber? (http://github.com/cpennington/pycucumber/tree/master) I am not sure how far along it is, but after checking out the links in the original comment, I think it might be interesting to track/contribute to.

Re: Python port

[maradydd.livejournal.com]

I hadn't, thanks for the link! I'm also going to have a look at Pyccuracy.