The whole process of creating a markup and developing tests is time consuming. And when it seems that the work is done, a new version of spec is released. What happens next? Of course there is a need of a new version of the test suite. New tests must be written and the old ones updated or even deleted.
The best way to start is to do the markup. This task can be devided into two subtasks:
-
transfer old markup from previous spec to the new spec (this is needed because many tests were already written, they are linked to spec id’s, reusing all possible tests is a good idea);
-
markup new and updated assertions.
Transferring the markup is simple enough to do it by hand:
-
Find the markup tag in the old spec.
-
Find the best place to insert the tag in the new version of spec.
-
Insert the tag.
If there are only 10 assertions – this work is a piece of cake. But if there are thousands it is a hard job that should be automated. The hardest part is to find a new proper place for markup tags. It’s hard just because the spec was changed. For JLS2 to JLS3 migration process the flollowing alrorithm was used:
Each assertion is rounded with html anchors. Both of them should be transferred using such algorithm.
Hint 2: algorithm should check that the second anchor should be positioned after the first one and not too far from it.
- Look at the text before and after tag in old spec. Find it in the new spec. If one of them
wasn’t changed – the answer is found. usually the length should be 1-2 sentances, at least 60 characters long. If none or several sentances found – skip this step. - Try to do the same as in (1), but remove all html tags from the near text that surrounds the tag. If none found – skip this step.
- Try the adopt algorithm, which tries to find similar text in the new spec.
a. Use steps (1) and (2), but desrease the length of the searching text in a loop until the sentance is found or the length is too short. The practical work showed that this number shouldn’t be less than 20.
b. If steps (1) and (2), or (3a) found several sentances increase the length of the text to search until the text is found in the new spec or the upper limit (f.e. 140 chars) is reached. Use hints to find the best matching text.
Adopt algorithm could be used with both ignoring html tags and taking advantage of them. Algorithm is valid for specs written in plain text, html or xml.
This algorithm was implemented in JLS2->JLS3 markup transfer tool. 84% of the markup tags were transfered automatically. The rest of them were done manually.
anchor, assertion, JLS, markup, markup transfer, spec, tests
There are sometimes facts, that connot be checked directly. Let’s say there is a refrigerator in a box, that may not be opened. Imagine refrigerator can be of 5 different colors – white, blue, red, black and green. The manual says that it is blue. But the only thing that is avaliable is a black-and-white photo of the refrigerator. The direct color check test is hard to make using just the photo (assume it is impossible). But one can defenately say (looking at the photo) that the refrigerator is either white or not. So the test doesn’t check the color itself, it actually tests if the color is white. 
