testresources

Have been working on an implementation of testresources , written by Robert Collins, for some time now, gotten it up and running, and now have good output in Jenkins...therefore, it seems about time to throw all that away in favor of a different implementation.

Although testresources appears much discussed in Python testing circles (see here, here and here), the documentation that exists on implementing it is...fairly sparse, to put it politely.  So over the last couple of days, I've waded hip deep into testresources package code.  I've found that the code itself is elegant and smart (I haven't tested the logic behind the various tools used in the implementation- digraph to graph, minimal spanning tree via Kruskals algorithm, etc.- but I do see that Mr. Collins put a great deal of effort into making it smart and powerful).

In summary, using the testresources package is a great way to allow access to expensive resources (in our case, a Selenium instance that fires up Firefox, imports a profile, and authenticates a user into our web interface) over multiple unit test cases in a suite that all require access to those resources.  The package even optimizes- <i>spelling it optimise to emfasise the British origin of the code, I guess</i>- the use of those resources by re-ordering test cases that call for the same resource to run all together so that, in theory, the resources persists only as long as there is a test case that is waiting to use it, then it is torn down via its native clean and tearDown methods (I say, "in theory," because if a test case changes the state of the resource and marks it <i>dirtied</i> the resource is destroyed and re-created- the order isn't smart enough to check and put any test cases that dirty the resource at the end of the queue so that the ones that can run happily one after the other don't get bogged down.)  To summarize, it does allow you to do some things that PyUnit doesn't allow, and it implements them very well.

Unfortunately, it also limits how you can structure your test cases (everything needs to be flat when you optimize it- one test suite consisting of all the test cases).  That means, you can't have nested suites (suites of suites), or sibling suites that contain cases that share resources.
This would work and be optimized with testresources:

<test suite>
     <test case 1>      - uses resource A
     <test case 2>
     <test case 3>      - uses resource A
     <test case 4>      - uses resource A
</test suite>

This doesn't:

<test suites>
     <test suite 1>
          <test case 1>      - uses resource A
          <test case 2>
          <test case 3>      - uses resource A
     </test suite 1>
     <test suite 2>
          <test case 4>      - uses resource A
     </test suite 2>

</test suites> 

So resource A would be created twice (at least), even though  it's essentially the same as the first structure (except that it uses a suite of test suites).

The next step, I think, is to try an implementation using the nose testing package.  More to follow.