TestsTracker is a web application for manual regression test case management that I made in response to an existing, workable, but imperfect process. While much has been made of the culture of testing in the world of unit tests and automation, manual regression tests seem to be a relic of the bad old days and don't see a lot of attention.
The application was designed to replace a collection of test cases written and reported on in Microsoft Excel spreadsheets. Theoretically, a feature's tests were covered by a single spreadsheet with worksheets covering different aspects of the testing. Regardless of the format or type of test performed, the test was documented in a template formatted approximately as follows:
+------------------------------------------------------------+
| Test Case Name Status |
+------------------------------+-----------------------------+
| Objective: | Notes: |
| | |
+------------------------------+-----------------------------+
| Test Step: | Explanation: |
| | |
+------------------------------+-----------------------------+
| Test Step: | Explanation: |
| | |
+------------------------------+-----------------------------+
| Test Step: | Explanation: |
| | |
+------------------------------+-----------------------------+
Where each table cell was a collection of merged cells with specific numbering, formatting, and styling. Initially, a collection of VBA macros had grown to encompass both data aggregation and reporting. This was eventually replaced due to the time required to run (which had reached over a half hour at times). It was replaced with a Python program using openpyxl which improved the run-time of the aggregation and reporting but punted on some of the more fundamental problems associated with the existing process. The rigid format of the spreadsheet was still unpleasant to work in for its tight coupling of test case data and the presentation format. Changes were never made to the process because it would require reworking the aggregation and reporting mechanisms.
Faced with those issues, I set about creating a system to solve the issues in the following priority:
I decided early on that the test case data was required to be both searchable and regular, something the existing system lacked (through the odd combination of MS Excel for test documentation and MS SharePoint for version control). Even with the data extracted from the Excel files, it is still possible to store the information in flat files, but for the ability to query on relevant attributes a database was chosen as the more standard interface for whatever front-end would access it.
The application is driven server side with Python using the Flask micro-framework along with SQLAlchemy, I use SQLite by default, but there is nothing precluding the use of any of the other databases that SQLAlchemy supports. The benefit of SQLite is that it comes packaged with Python and requires no additional setup or installation1.
I strove to make the API as obvious as possible, but refrain from calling it REST because I've spent an embarrassing amount of time reading things like:
A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types.
which set my head spinning. I will instead call the API resource-oriented and avoid any discussion of Richardson's Maturity Model or Maturity Heuristics2.
The key resource within the application is based on the following database model:
class TestRecord(db.Model):
id = db.Column(db.String(32), primary_key=True)
user = db.Column(db.String(100))
suite = db.Column(db.String(200))
testcase = db.Column(db.String(200))
notes = db.Column(db.Text)
def __init__(self, user, suite, testcase, notes):
self.id = uuid4().hex
self.user = user
self.suite = suite
self.testcase = testcase
self.notes = notes
Which effectively captures the same information as the spreadsheet template, without any regard for the eventual presentation format. Rather than try to define my own primary key for each record, I defer to Python's excellent standard library and use a UUID which is generated for each new record upon instantiation (as an interesting aside, UUIDs have 2128 possible values, which is mind-bogglingly large3). The UUID guarantees uniqueness at the expense of obtuse resource names, which was acceptable given the audience for direct interaction with the API outside of the application front-end (realistically, only me).
The standard HTTP verbs (GET, PUT, POST, DELETE) are supported,
predictable actions on each of those resources with the data serialized as JSON
for ease of use. As an example, the following request:
$ curl http://localhost:5000/tests
returns a JSON object with a single array4] of all test records:
{
"results": [
{
"id": "98f957ebcab647e5a73a5e3458ff03cd",
"notes": "text field",
"suite": "string field",
"testcase": "string field",
"user": "string field"
},
{
"id": "db142dbb3d374a79b9e7aebae6f5cf8e",
"notes": "text field",
"suite": "string field",
"testcase": "string field",
"user": "string field"
}
]
}
and a request to
$ curl -i -X DELETE http://localhost:5000/tests/db142dbb3d374a79b9e7aebae6f5cf8e
returns
HTTP/1.0 204 NO CONTENT
Content-Type: application/json
Content-Length: 0
Server: Werkzeug/0.9.6 Python/2.7.8
after successfully deleting the record. This should serve to demonstrate the entire API is (hopefully) as unsurprising as possible - the back-end intentionally enforces little constraint and instead delegates to the front-end in order to allow for different operations which I might not have designed for originally (perhaps batch loading or export facilities).
This is getting obnoxiously large at this point.