Is Test Driven Development in Drupal possible?

I was searching for a way to make test writing for Media a little less painful.  I use TDD in Ruby, Python and even in PHP projects I did before coming to Drupal (5 years ago).  It pains me that I'm unable to find a way to do it in Drupal.  I read a fantastic writeup by a dedicated TDD engineer trying to switch to Drupal

"One minute per test case when I'm expecting to easily more than a hundred test cases feels insane."

I personally agree.  And I'm not going to spend a few hours setting up mysql on a RAM disk just to go from 3 minute runs to 1.5 minute runs.  The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have.  

So if we're going to rely on functional tests, we can kiss TDD goodbye.  What is the long term solution here then?  Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this?  The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound.

Maybe the answer is Simpletest 2 which allows you to run on a "dirty" environment.  This certainly makes the tests faster at the expense of accuracy.  But if you do that while developing it will help.  I'm going to try and start using that for Media tests.  I'll update you on the results.

Anyone have any other ideas here?

 

Tags: 

Comments

I think a majority of folks

Kevin's picture

I think a majority of folks have ignored TDD or writing tests in general because it takes too long (to either write OR execute).

To be efficient with it, I believe writing tests should be as natural as writing hooks, and not require large lengths of time to test out little changes (or big ones). Speed is a big factor.

Drush folks are just as frustrated as you are.

Check out the video below. There is work started on getting phpunit working on drupal and they claim to have 35% test coverage. That's a start, and for your own tests it might be fairly simple to start using it, instead of simpletest.

http://chicago2011.drupal.org/sessions/advanced-drush

You made some splendid

malingbowler's picture

You made some splendid factors there. I do a lookup concerning the subject and found out most individuals will concur jointly with your blog. Demonsys.com

I am currently working on an

Android tablet games's picture

I am currently working on an assignment and I have been exploring your blog for a few hours. Thank you for your post it proved helpful for me

Android tablet games

Wonderful being going to your

malingbowler's picture

Wonderful being going to your blog once again, it's been weeks to me. This report in which i am silently laid for such a long time. I would like this information to complete my own project within the school, and possesses exact same topic using your.. read more here

Really loved reading your

malingbowler's picture

Really loved reading your blog. It was very well authored and easy to undertand. Unlike additional blogs I have read which are really not tht good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he ejoyed it as well recording webinars

One thing I like coming to

malingbowler's picture

One thing I like coming to this forum is that there is something that you can learn each time or some tip that will help you to make your work easier and simpler!! But today I think things were a little Today we have been given a small revision on File Management but I where not only the techniques of working with filter declaration such as caching, session etc but also do several parent classes other simple techniques as well!! phen375 scam

Butler

Thankfully the Butler project is taking this into consideration... making contexts mockable will facilitate unit testing. But as far as testing framework, there would need to be something else more suited to TDD.

What are the actual problems?

I can't tell what your actual problems are besides:
1. it runs too slow
2. it can't test against the 'current' enviornment
3. you don't like it?

Is there something more concrete behind the problems writing tests for Media? It would probably help if the QA testbot could properly test modules with dependencies since you could rely on using that for running tests, and that's something that people are working on.

Hi Dave,

Jacob Singh's picture

Hi Dave,

Thanks for stopping by!

The main issue IMO is speed.  I don't like writing tests after the code is done.  TDD means the test *drives* the development, as in the test comes first / durring.  Could you imagine developing anything and waiting even a minute (and even running one test usually takes 2-3 minutes on my laptop) every time you wanted to iterate?  I know you're a genious and all, but even you must have typos or want to see how something behaves every now and then.

So yeah, I think speed is the main issue.  And most of that is re-installing Drupal for every test case.  I know, functional testing is not reliable otherwise.  I'm not saying I have a solution, I'm saying I have a problem :)

 

Problem with Simpletest

@Dave, I think the issue is not with Simpletest. ST is great for it's intended purpose and it has been awesome for Drupal and will continue to make Drupal better. However, it is not really a unit testing framework. (I'm thinking jUnit as that is what I'm familiar with.)

I'm hoping that the solution involves extending Simpletest or using something that can easily coexist with ST.

workaround

This is a serious limitation of our testing framework - every testXXX() method triggers a complete re-install of Drupal, from scratch. This essentially means that every hook_install() becomes part of a giant monolithic and expensive text fixture. To make things worse, the UI offers no way to select and run an individual test function (only "groups" declared in getInfo()).

One way of working around this is to create only one test method, and using it as a wrapper for other functions. It's a bit clumsy but you can speed up your test runs tremendously. I use this approach in the Secure Pages module and it cut test execution time from 3 minutes to about 20 seconds.

http://drupalcode.org/project/securepages.git/blob/HEAD:/securepages.test#l27

The main drawback is each sub-test needs to be careful to clean up any system-wide settings it may have changed, so as not to impact subsequent tests.

Use testing profile?

Have you tried using the $profile = 'testing' in your classes? That's supposed to improve speed by not having so many core modules enabled every setUp() run compared to the standard profile.

Yeah SimpleTest has moved far

Yeah SimpleTest has moved far from being a unit testing framework. It's more of a functional testing framework and I don't think it can be adapted to do both without major work.

Generally I write features with the test in mind, and then write the test after a fair chunk of code is written to verify that I did it correctly. Generally I can deal with a 1-2 minute wait for the tests since I'm doing it in chunks and I only select the tests that I'm currently working on. I maybe do a full test review every week or so, or if the qa.drupal.org testbot can do it, I let it handle test runs for me after I commit & push.

How about DrupalUnitTestCase?

David Rothstein's picture

Although the Drupal testing framework isn't primarily designed for unit testing, if you have code that's truly unit-testable, it is actually possible to write unit tests for it that run very fast. All you do is have your test class extend DrupalUnitTestCase rather than DrupalWebTestCase.

There are limitations on what kind of code you can run during those tests (since you're not working with an actual Drupal installation at that point), so it certainly won't work for everything. But in cases where it does work, the tests run lightning-fast and test-driven development is definitely possible.

Yeah, I realize that, but as

Jacob Singh's picture

Yeah, I realize that, but as I stated above:

"The only answer is of course unit testable code which for any substantial module means a unit testable framework, which we really don't have. "

and

" Can we make D8 more pluggable so we can use mocks around cleaner interfaces to avoid this?  The issue is that Drupal's USP is you can mess with anything anywhere… as soon as you start introducing mocks, you kinda lose that which means the tests aren't as sound."

Everytime I've sat down to write unit tests in Drupal, my code ends up become so convoluted to avoid touch any piece of Drupal itself that I'm not testing anything.  Couple that with the fact that because we are not using classes, I can't extend and mock up calls that would use Drupal.

You can unit test stuff like regexes or really basic file I/O parsing operations, but anything that involves logiic probably involves some other subsystem somewhere.

There are limitations on what

iomundo's picture

There are limitations on what kind of code you can run during those tests (since you're not working with an actual Drupal installation at that point), so it certainly won't work for everything. iomundo

It is evidently that essay

jamesdoef's picture

It is evidently that essay writing services can advice the notices close to dissertation service moreover, men can very easily buy a term paper or thesis service just about this good post izrada web stranica

Took me time to read all the

Web Directory's picture

Took me time to read all the comments, but really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It's always nice when you can not only be informed, but also entertained! I'm sure you had fun writing this article

Web Directory

Great information, I’ve been

funny valentine quotes's picture

Great information, I’ve been reading about this topic for one week now for my exams in school and thank God I found it here in your blog. I had a great time reading this .
<a href="http://www.funnyjokes7.com/category/funny-valentine-quotes/" rel="dofollow">funny valentine quotes</a>

This is my first opportunity

jhonbothaw's picture

This is my first opportunity to visit this website. I found some interesting things and I will apply to the development of my blog. Thanks for sharing useful information. Try Here

Wonderful work. This

Wonderful work. This particular information will be very useful for a lot of people. I really like the blog significantly. I don’t understand home what to express except that We've enjoyed browsing.

Took me time to read all the

slots's picture

Took me time to read all the comments, but really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It's always nice when you can not only be informed, but also entertained! I'm sure you had fun writing this article. slots

I wonder how you got so good.

carlaimre's picture

I wonder how you got so good. This is really a fascinating blog, lots of stuff that I can Get into. One thing I just want to say is that your Blog is so perfect! cell phone spy

The blog is very well

Mike Turn's picture

The blog is very well implemented, I love the big font you utilize also the topic of this article is very knowledgeable. Thanks for sharing. international education

You have been great, please

Chris Harris's picture

You have been great, please keep updating us on the latest, really love everything that yo have been sharing around..
Chris Harris
Premium Recovery

This is exactly what I was

pds behandeling's picture

This is exactly what I was looking for. Thanks for sharing this great article! That is very interesting Smile I love reading and I am always searching for informative information like this! Visit my site http://wegmetpds.be/blog/pds-behandeling/

Nice blog

John Lavy's picture

This is a cool blog to visit often. I love the way you font the comment with big font. And also your article is a very good one. Keep up a good work buddy custom small pools

I’m very little directly i

Jennry's picture

I’m very little directly into studying, however by some means I obtained to learn numerous content inside your webpage. It’s amazing just how interesting it is that i can go to anyone usually. where to eat in singapore?

I will bookmark your blog and

home painters brisbane's picture

I will bookmark your blog and have my friends check up here often. I am quite sure they will learn lots life coaching for women of new stuff here than anybody else. Thanks for sharing this information.