Difference between revisions of "SoC test framework"

Revision as of 16:04, 21 May 2009

This is one of the projects to be implemented during the Google Summer of Code 2009.

Student: Sean Clark, Mentor: Dirk Haun
Project repository

Project info and advancement during Summer of Code

• first decision: We are going to use PHPUnit.
• This will be a separate project, to be run against the Geeklog repository. So we're starting with a fresh repository (instead of branching the Geeklog repo).

(more: TBD)

Pre-Summer of Code idea discussion & preparation

Incentive

Very little testing on Geeklog is currently done automatically. As the complexity of the software is ever increasing, it's not enough to rely on manual tests done by the developers (and, by extension, the community during beta and release candidate phases) any more.

So the goal of this project is to implement test framework(s) to allow for automated tests of Geeklog and its features.

Details

There are two aspects that we'd like to see tested, which may actually make it possible to have this split into two projects:

• tests of the code on class / function level
• tests of the front-end functionality available through the browser

At the code level, unit tests are required to make sure the code behaves as expected at its interface (API) so that after making changes to the internals, the API still behaves the same.

Unit testing is usually associated with object oriented code, which may sound like a problem given that Geeklog is not fully object oriented. However, unit tests can also be written for the many collections of functions in system/lib-*.php and lib-common.php.

Testing the functionality implemented in, say, admin/story.php (the Admin panel that provides access to all stories on a site) is probably best done through a tool that tests the web frontend.

Frameworks

There already exist numerous test frameworks for PHP code and for web applications. So unless there's a very good reason not to pick one of those, we would suggest looking into existing frameworks (and adapt them, if necessary).

Here's an incomplete list of test frameworks. We expect the student to research more options before making a choice:

• PHP Tests (*.phpt) as used by the PHP project
• PHPUnit
• Selenium

Open source solutions would be preferred, although there are some commercial frameworks that may be worth looking into.

Level of Difficulty

low to medium

The focus in this project lies on picking the proper tool (i.e. test framework) and then demonstrating that it does the job by implementing tests for a representative portion of the code / functionality. We certainly don't expect complete coverage of all of Geeklog in just one summer. This project is about laying the groundwork.