Is Test Driven Development in Drupal possible?
I was searching for a way to make test writing for Media a little less painful. I use TDD in Ruby, Python and even in PHP projects I did before coming to Drupal (5 years ago). It pains me that I'm unable to find a way to do it in Drupal. I read a fantastic writeup by a dedicated TDD engineer trying to switch to Drupal
"One minute per test case when I'm expecting to easily more than a hundred test cases feels insane."
I personally agree. And I'm not going to spend a few hours setting up mysql on a RAM disk just to go from 3 minute runs to 1.5 minute runs. The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have.
So if we're going to rely on functional tests, we can kiss TDD goodbye. What is the long term solution here then? Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this? The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound.
Maybe the answer is Simpletest 2 which allows you to run on a "dirty" environment. This certainly makes the tests faster at the expense of accuracy. But if you do that while developing it will help. I'm going to try and start using that for Media tests. I'll update you on the results.
Anyone have any other ideas here?
Addthis
Comments
I think a majority of folks
I think a majority of folks have ignored TDD or writing tests in general because it takes too long (to either write OR execute).
To be efficient with it, I believe writing tests should be as natural as writing hooks, and not require large lengths of time to test out little changes (or big ones). Speed is a big factor.
Drush folks are just as frustrated as you are.
Check out the video below. There is work started on getting phpunit working on drupal and they claim to have 35% test coverage. That's a start, and for your own tests it might be fairly simple to start using it, instead of simpletest.
http://chicago2011.drupal.org/sessions/advanced-drush
Butler
Thankfully the Butler project is taking this into consideration... making contexts mockable will facilitate unit testing. But as far as testing framework, there would need to be something else more suited to TDD.
What are the actual problems?
I can't tell what your actual problems are besides:
1. it runs too slow
2. it can't test against the 'current' enviornment
3. you don't like it?
Is there something more concrete behind the problems writing tests for Media? It would probably help if the QA testbot could properly test modules with dependencies since you could rely on using that for running tests, and that's something that people are working on.
Hi Dave,
Hi Dave,
Thanks for stopping by!
The main issue IMO is speed. I don't like writing tests after the code is done. TDD means the test *drives* the development, as in the test comes first / durring. Could you imagine developing anything and waiting even a minute (and even running one test usually takes 2-3 minutes on my laptop) every time you wanted to iterate? I know you're a genious and all, but even you must have typos or want to see how something behaves every now and then.
So yeah, I think speed is the main issue. And most of that is re-installing Drupal for every test case. I know, functional testing is not reliable otherwise. I'm not saying I have a solution, I'm saying I have a problem :)
Problem with Simpletest
@Dave, I think the issue is not with Simpletest. ST is great for it's intended purpose and it has been awesome for Drupal and will continue to make Drupal better. However, it is not really a unit testing framework. (I'm thinking jUnit as that is what I'm familiar with.)
I'm hoping that the solution involves extending Simpletest or using something that can easily coexist with ST.
workaround
This is a serious limitation of our testing framework - every testXXX() method triggers a complete re-install of Drupal, from scratch. This essentially means that every hook_install() becomes part of a giant monolithic and expensive text fixture. To make things worse, the UI offers no way to select and run an individual test function (only "groups" declared in getInfo()).
One way of working around this is to create only one test method, and using it as a wrapper for other functions. It's a bit clumsy but you can speed up your test runs tremendously. I use this approach in the Secure Pages module and it cut test execution time from 3 minutes to about 20 seconds.
http://drupalcode.org/project/securepages.git/blob/HEAD:/securepages.test#l27
The main drawback is each sub-test needs to be careful to clean up any system-wide settings it may have changed, so as not to impact subsequent tests.
Use testing profile?
Have you tried using the $profile = 'testing' in your classes? That's supposed to improve speed by not having so many core modules enabled every setUp() run compared to the standard profile.
Yeah SimpleTest has moved far
Yeah SimpleTest has moved far from being a unit testing framework. It's more of a functional testing framework and I don't think it can be adapted to do both without major work.
Generally I write features with the test in mind, and then write the test after a fair chunk of code is written to verify that I did it correctly. Generally I can deal with a 1-2 minute wait for the tests since I'm doing it in chunks and I only select the tests that I'm currently working on. I maybe do a full test review every week or so, or if the qa.drupal.org testbot can do it, I let it handle test runs for me after I commit & push.
How about DrupalUnitTestCase?
Although the Drupal testing framework isn't primarily designed for unit testing, if you have code that's truly unit-testable, it is actually possible to write unit tests for it that run very fast. All you do is have your test class extend DrupalUnitTestCase rather than DrupalWebTestCase.
There are limitations on what kind of code you can run during those tests (since you're not working with an actual Drupal installation at that point), so it certainly won't work for everything. But in cases where it does work, the tests run lightning-fast and test-driven development is definitely possible.
Yeah, I realize that, but as
Yeah, I realize that, but as I stated above:
"The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have. "
and
" Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this? The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound."
Everytime I've sat down to write unit tests in Drupal, my code ends up become so convoluted to avoid touch any piece of Drupal itself that I'm not testing anything. Couple that with the fact that because we are not using classes, I can't extend and mock up calls that would use Drupal.
You can unit test stuff like regexes or really basic file I/O parsing operations, but anything that involves logiic probably involves some other subsystem somewhere.
Hi there
It's great post, can I republish this post on my blog if I link back to you?How to know if you are pregnant
seems to be possible. I am
seems to be possible. I am sure that Drupal will accept major changes! charter yacht
There are limitations on what
There are limitations on what kind of code you can run during those tests (since you're not working with an actual Drupal installation at that point), so it certainly won't work for everything. iomundo
Gangaur Realtech is a
Gangaur Realtech is a professionally managed organisation specializing in real estate services where integrated services are provided by professionals to its clients seeking increased value by owning, occupying or investing in real estate. commercial projects in gurgaon | plots in dharuhera | commercial space in gurgaon
Exceeding assemblage you got
Exceeding assemblage you got here. I've been datum virtually this master for one week now for my composition in change and communicate God I recovered it here in your writing. I had a broad theorisation representation this .
Acheter monster beats by dre
Acheter monster beats by dre headphones.Our par casque dr dre à la vente sont chauds
beats by dre
Post new comment