Framework for automatic testing
Tests on cloud9
We need to fully validate the code each time it is changed.
Computers We can dedicate two machines to this testing, cloud9 and fog, and can buy more if needed. Cloud9 is an 8-core 64 bit linux machine while fog is 2-core 32 bit. Both machines are on the physics linux cluster. We have a special user account, cloudtests, set up for this testing.
tests now done The script run_daily_tsuite.pl in the root of ~cloudtests will run the auto and slow test suites on 8 processors, using a nice level of 10. If there were no updates to the repository, the test suite will not be run, and no mail should be sent.
Which tests on which days is controlled by the script run_all_cloud9.pl.
Reporting The ~cloudtests/.forward file controls who gets emails produced by daily runs. A detailed report could be deposited in ~cloudtests/public_html so that reports can be easily viewed at http://www.pa.uky.edu/~cloudtests.
Could a web-based tool like Google refine be used to harvest information and generate reports?
"crontab -e" has
02 00 * * * /home66/cloudtests/run_all_cloud9.sh
Open source frameworks
There is a balance between setting things up from scratch ourselves and using an external off-the-shelf system. The latter option may provide additional capability relatively easily, at the cost of learning how to set up and use the system. Lighter weight frameworks may offer a good compromise between saving effort and ease of configurability (depending on implementation language).
Good summaries in following Wikipedia articles
- Fowler's book on continuous integration
- Hudson is a more recent popular Java-based alternative
- BuildBot is a relatively lightweight system, scriptable in Python