Generating blog posts

Having fun with JUnit and Acceleo

After playing with JUnit 4 and its new features during the last few weeks, I wondered if a unit testing framework for Acceleo would be interesting. So I tried to create something to test a code generatr easily. I started it as a small prototype but after spending most of this rainy sunday on it, here are my conclusions.

I choose to build that testing framework on top of JUnit 4 because JUnit is compatible with countless tools and since there are providing a simple API to extend it, there is no need to reinvent the wheel. My solution is based on three different parts, a suite, a runner and some statements:

• The AcceleoSuite is a JUnit suite which will discover all the available uri fragments and then create a runner for each fragment available.
• The AcceleoRunner is a JUnit runner which will determine the methods that need to be tested and it will create a test for each of those methods.
• The AcceleoTemplateStatement and AcceleoQueryStatement are in charge of the execution of the test. They will first create a helper object in order to simply the test for the end user and then they will call the method to test with this helper object as a parameter. They are two statements, one for each kind of tests.

That architecture is based on some observations of the Theory and the Parameterized test systems. With this system, the user has to provide some informations in order to run the test:

• The path of the Acceleo module to test
• The uri fragments of the EMF model elements to use
• The name of the template / query to execute

In order to provide those information, I have chosen to create several annotations.

• @ModuleTest
• @ModelElement(s)
• @TemplateTest / @QueryTest

Finally, to make it easy to create and maintain those tests, I choose to create some assertion methods based on the Harmcrest framework. You can see here a screenshot of what an Acceleo unit test should looks like in the end.

After several hours of coding, testing and tweaking I came to some conclusions about that unit testing framework:

• The Acceleo engine appears to not be able to execute a query and return the result. The starting point of a generation is always a template but in order to have an effective testing framework, it must have the ability to execute a query and process the result. It may require some new features in the Acceleo engine API. I’ll have to ask Laurent about it.
• This framework needs to be able to run in a stand alone mode. There is no way that the user will launch a new instance of Eclipse for its unit tests. The parser is stand alone, the engine is stand alone, the unit testing has to be stand alone. In stand alone, I encountered several problems while loading some of my models. This will be a problem for people with tricky metamodels that needs a special loading policy (yes UML, I’m thinking about you and your profiles). In order to have some room for improvement I have created another annotation, @LoadedModelElements which let the user loads the model elements and then just give them to the framework. This annotation is not taken into account right now.
• It definitely needs an UI (EEF ?). Sure, some of the users of Acceleo can write a test like in the example easily, but quickly if you really want to test a real model with thousand of elements and tricky URIs it becomes impossible to manage. As such, a small wizard is needed. In that wizard, we could load a model and select its elements in a treeviewer and then load an Acceleo module and select the templates and queries  to test. After this we could easily generate a basic test class for this module. I am not sure the uri fragment is the best way to identify the elements of the model that we want to use for our generation, I will have to work on that later.
• I also started to think about some other testing features that could be useful and I think that a trace system would be very useful. By trace system, I mean a new engine that would generate the code and warn a listener for all the instructions that are visited in the module. This trace would be very useful for the code coverage system but it would also be useful order to create a test in which we could write something like this: “assertThatNumberOfCallToTemplateNamed(“generate”, is(2))”, just like with the mockito framework in Java. I am pretty sure that the algorithm is already there in order to make the profiler work but I am not sure if it is open enough for me to plug some listener.

In order to conclude this post, I would say that I am not far from a working prototype for template testing, it just needs some work in order to be stand alone. But to have a good unit testing framework, it will definitely require more work especially on the API of the engine.