Is Test Driven Development in Drupal possible?

Submitted by Jacob Singh on

I was searching for a way to make test writing for Media a little less painful.  I use TDD in Ruby, Python and even in PHP projects I did before coming to Drupal (5 years ago).  It pains me that I'm unable to find a way to do it in Drupal.  I read a fantastic writeup by a dedicated TDD engineer trying to switch to Drupal

"One minute per test case when I'm expecting to easily more than a hundred test cases feels insane."

I personally agree.  And I'm not going to spend a few hours setting up mysql on a RAM disk just to go from 3 minute runs to 1.5 minute runs.  The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have.  

So if we're going to rely on functional tests, we can kiss TDD goodbye.  What is the long term solution here then?  Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this?  The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound.

Maybe the answer is Simpletest 2 which allows you to run on a "dirty" environment.  This certainly makes the tests faster at the expense of accuracy.  But if you do that while developing it will help.  I'm going to try and start using that for Media tests.  I'll update you on the results.

Anyone have any other ideas here?

 

Tags: 

Comments

I think a majority of folks

I think a majority of folks have ignored TDD or writing tests in general because it takes too long (to either write OR execute).

To be efficient with it, I believe writing tests should be as natural as writing hooks, and not require large lengths of time to test out little changes (or big ones). Speed is a big factor.

Wonderful being going to your

Wonderful being going to your blog once again, it's been weeks to me. This report in which i am silently laid for such a long time. I would like this information to complete my own project within the school, and possesses exact same topic using your.. read more here

Really loved reading your

Really loved reading your blog. It was very well authored and easy to undertand. Unlike additional blogs I have read which are really not tht good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he ejoyed it as well recording webinars

One thing I like coming to

One thing I like coming to this forum is that there is something that you can learn each time or some tip that will help you to make your work easier and simpler!! But today I think things were a little Today we have been given a small revision on File Management but I where not only the techniques of working with filter declaration such as caching, session etc but also do several parent classes other simple techniques as well!! phen375 scam

What are the actual problems?

I can't tell what your actual problems are besides:
1. it runs too slow
2. it can't test against the 'current' enviornment
3. you don't like it?

Is there something more concrete behind the problems writing tests for Media? It would probably help if the QA testbot could properly test modules with dependencies since you could rely on using that for running tests, and that's something that people are working on.

Hi Dave,

Hi Dave,

Thanks for stopping by!

The main issue IMO is speed.  I don't like writing tests after the code is done.  TDD means the test *drives* the development, as in the test comes first / durring.  Could you imagine developing anything and waiting even a minute (and even running one test usually takes 2-3 minutes on my laptop) every time you wanted to iterate?  I know you're a genious and all, but even you must have typos or want to see how something behaves every now and then.

So yeah, I think speed is the main issue.  And most of that is re-installing Drupal for every test case.  I know, functional testing is not reliable otherwise.  I'm not saying I have a solution, I'm saying I have a problem :)

 

Problem with Simpletest

@Dave, I think the issue is not with Simpletest. ST is great for it's intended purpose and it has been awesome for Drupal and will continue to make Drupal better. However, it is not really a unit testing framework. (I'm thinking jUnit as that is what I'm familiar with.)

I'm hoping that the solution involves extending Simpletest or using something that can easily coexist with ST.

workaround

This is a serious limitation of our testing framework - every testXXX() method triggers a complete re-install of Drupal, from scratch. This essentially means that every hook_install() becomes part of a giant monolithic and expensive text fixture. To make things worse, the UI offers no way to select and run an individual test function (only "groups" declared in getInfo()).

One way of working around this is to create only one test method, and using it as a wrapper for other functions. It's a bit clumsy but you can speed up your test runs tremendously. I use this approach in the Secure Pages module and it cut test execution time from 3 minutes to about 20 seconds.

http://drupalcode.org/project/securepages.git/blob/HEAD:/securepages.test#l27

The main drawback is each sub-test needs to be careful to clean up any system-wide settings it may have changed, so as not to impact subsequent tests.

Yeah SimpleTest has moved far

Yeah SimpleTest has moved far from being a unit testing framework. It's more of a functional testing framework and I don't think it can be adapted to do both without major work.

Generally I write features with the test in mind, and then write the test after a fair chunk of code is written to verify that I did it correctly. Generally I can deal with a 1-2 minute wait for the tests since I'm doing it in chunks and I only select the tests that I'm currently working on. I maybe do a full test review every week or so, or if the qa.drupal.org testbot can do it, I let it handle test runs for me after I commit & push.

How about DrupalUnitTestCase?

Although the Drupal testing framework isn't primarily designed for unit testing, if you have code that's truly unit-testable, it is actually possible to write unit tests for it that run very fast. All you do is have your test class extend DrupalUnitTestCase rather than DrupalWebTestCase.

There are limitations on what kind of code you can run during those tests (since you're not working with an actual Drupal installation at that point), so it certainly won't work for everything. But in cases where it does work, the tests run lightning-fast and test-driven development is definitely possible.

Yeah, I realize that, but as

Yeah, I realize that, but as I stated above:

"The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have. "

and

" Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this?  The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound."

Everytime I've sat down to write unit tests in Drupal, my code ends up become so convoluted to avoid touch any piece of Drupal itself that I'm not testing anything.  Couple that with the fact that because we are not using classes, I can't extend and mock up calls that would use Drupal.

You can unit test stuff like regexes or really basic file I/O parsing operations, but anything that involves logiic probably involves some other subsystem somewhere.

Took me time to read all the

Took me time to read all the comments, but really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It's always nice when you can not only be informed, but also entertained! I'm sure you had fun writing this article

Web Directory

Great information, I’ve been

Great information, I’ve been reading about this topic for one week now for my exams in school and thank God I found it here in your blog. I had a great time reading this .
<a href="http://www.funnyjokes7.com/category/funny-valentine-quotes/" rel="dofollow">funny valentine quotes</a>

Took me time to read all the

Took me time to read all the comments, but really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It's always nice when you can not only be informed, but also entertained! I'm sure you had fun writing this article. slots