Generating blog posts

A few hours with Orion

Since I am currently working on the Javascript/HTML/CSS prototype for the EclipseCon Modeling tutorial during which we will see how to use a model driven approach to build web applications, I have decided to try Orion.

For those who don’t know Orion, it is one of the latest projects of the Eclipse foundation and it can be sum up as a web based development environment. Right know, the Orion team is focused on building a development environment for web developers which is logical since they want to use Orion to build Orion. So I tried Orion 0.4 M2 during a few hours to work on this Javascript project.

Orion was surprisingly easy to install and to start, with a file to unzip and an “.exe” file to double click on. Once launched, you can clone a git repository and start working very quickly. In Orion, a “regular Eclipse workbench” is represented by several pages that you can open in different tabs and while you can find most of the features needed to work, you can’t view at the same time the navigator and the editor so I had to spend most of my time switching from one tab to another. I believe it is currently the main problem with approach used in Orion.

As for editing part, the Javascript editor provided by Orion detects more problems than the editor in Eclipse and even if there are some false positive due to the complexity of some frameworks available in Javascript, I was impressed by the overall quality of the editor. Combined with the awesomeness of the developer tools of Chrome, it was a great coding experience. Being able to host the web application that you are working on directly on Orion is the cherry on the cake. Few clicks here and there and it’s running.

I had some issues with the stability of Orion with 4 crashes in 4 hours and with Chrome, once Orion crashes, all the tabs using Orion crash at the same time. Each time, I just had to do a simple F5 on each tab to reload them and I was ready to work and every single time without loosing anything. So even with those crashes, it is a very reliable tool. This was also possible by having this separation of each tool in its on tab. While this approach makes me spend a lot of time switching tabs, it also has some neat advantages. Since Orion has an URL for pretty much anything, I could get my editor back by opening a tab on its URL (something like this: “http://localhost:8080/edit/edit.html#/file/e/path_to_my_file”).

The git tooling is good, it allows me to work with my existing repository very quickly even if the user interface was a bit disturbing, everything needed was there so I think it’s just a matter of time to get use to it. I had a minor issue with it as it did not seem to find my ssh key contrary to the good ol’ git command line tool. It must have something to do with the fact that my ssh key is not in the default folder for ssh keys. So once in a while, I had to do a “git push origin master” from the terminal.

Edit: copying the ssh key directly in the dialog with the password works.

To conclude, I would say that working with Orion was a very good experience (especially for a 0.4 M2 version) and to work with Javascript/HTML/CSS most of the tools that are missing in Orion (ex: debugger) can be found in the developer tools of Chrome (or Firefox). There are still lot of things that can be improved like the outline which is not very useful right now and the profile management which is also quite weak but for the editing part it’s a really good experience minus an interface which feels a bit raw.

Having fun with Chrome’s APIs

While working on the future Acceleo tutorial that will occurred during EclipseCon 2012, I have to work with several javascript frameworks and with the API of the Chrome browser. Of course, as usual, it’s a nice opportunity to see what can be done with those.

I believe that I can use that opportunity to build a small extension for those who want to find Acceleo related content on internet quickly. I’ll have to work on the design and I hope that I will be able to integrate inside real time news, a good opportunity to have a look at Twitter APIs. Here’s a small screenshot of a first version realized in a few minutes.

Quick and dirty maven repository for Acceleo

You want to use Acceleo jars in a stand alone project with maven? You don’t know how to host Acceleo jars in a maven repository? Well, it’s simple:

1- create a local maven repository

2- put Acceleo jars inside

3- push it on github :)

So while I’m waiting for the Eclipse Foundation to put the content of the release train online, I think that I’ll put everything needed for Acceleo on this repository. If you want to use it, have a look at this screenshot.

Right now there are only the jars of the latest build (I’ll have to see if I push the jars of the latest build automatically easily), but I’ll put the different stable release along with Acceleo dependencies. Of course, that’s a temporary solution but if you need a quick and dirty maven hosting solution, it works :)

UML to Java Generator for Acceleo

Scala based Acceleo launcher

Scala is a great functional and object oriented programming language. It can be very easily integrated in a Java based application. When you create an Acceleo generator, Acceleo generates a Java class to launch your generation easily. You can use this class from any Scala based application without any difficulties but if you really want to have a pure Scala based application, why would you want to manipulate this Java class, why not replace it with Scala?

So while I was still playing with Scala, I’ve decided to create an Acceleo launcher and its matching UI plugin directly in Scala. For those who are not familiar with Scala, not only can you call Java code from Scala, you can also inherit from Java classes and much more. I invite you to have a look at this great language on its official website.

For those of you who were looking for an Acceleo generator to generate Scala source code, you should definitely have a look at this. If you just want to use Acceleo in a Scala based environment, you can find the source code of the Scala based Acceleo launcher and its UI plugin under the open source EPL licence on github.com.

For more information on Acceleo, you can follow me on twitter or google+.

Scala generation with Acceleo 3.1

I started Scala few months ago and since then I’m loving it. At the beginning, I’ve started by creating basic Scala applications in order to learn the language and after that I moved to small prototypes that I will present and improve in the months to come.

Quickly I was impressed by the structure of the Scala language and I started to look for a Scala metamodel in order to study the concepts of this language. Without any surprise, I didn’t found anything except a few people who were also looking for a Scala metamodel. Well since I am familiar with modeling technologies, I had to try to create a metamodel of Scala by myself, and you know me, if I have a metamodel, I have to generate code out of it.

This generator uses a PSM to generate scala source code. You can create the Scala model with the good old tree based EMF editor. For those who want to have fun, you can also grab an evaluation version of Obeo Designer and try to create diagrams for Scala. In less than five min, you can already have a small class diagram editor.

I have mainly focus my work on an Acceleo generator that I’ve recently updated to be conform with the best practices defined for Acceleo and I also switched to the latest version of Acceleo released with Eclipse 3.7 Indigo in order to use some of the brand new features coming with Acceleo 3.1.

You can easily build the generator very easily, thanks to maven and tycho, with a simple command: “mvn clean package”. After that, you will have an Eclipse update site in order to deploy the generator in an Eclipse instance of your choice.

This generator can generate Scala classes, objects and traits as you can see on the following screenshots.

The metamodel, the generator and their source code are available on Github under the open source EPL licence, feel free to test, use or even fork them as you want. For more information on Acceleo, you can follow me on twitter or google+.

Experimenting with Scala and Acceleo

During the last few days, I have been prototyping some new stuff but contrary to some of my previous experiments, this time I will not try to create something new, I’ll try to improve an existing component of Acceleo, the Acceleo parser.

The current situation

For this experiment, I choose to concentrate my effort on the analysis of the text. For those who are not familiar with parsing and more specifically the Acceleo parser, here is how its working:

1. Acceleo parses the text in a given file to create an abstract syntax tree representing the different parts of the text (module declaration, templates queries, if blocks, etc)
2. Then Acceleo resolves the links between modules (module A is extending module B, module B is importing module C, etc)
3. After that, Acceleo uses the MDT OCL project to parse the OCL expressions (guards, body of the queries and OCL expressions in templates, etc)

During all those steps, we are also detecting some potential errors here and there. In the first part, we detect the most important errors to quickly find out invalid modules, then in the next steps, we will look for logical errors like undefined operations, duplicated template declaration, or even invalid metamodels. This prototype will for now only try to replicate the first step with the analysis of the text to produce an AST.

The classes used by Acceleo to parse the text into an unresolved AST have 1107 + 1781 + 373 + 440 + 281 = 3982 lines of code. This does not take into account all the code dedicated to the parsing of OCL expression and the resolution of the links between modules. It can detect a huge number of errors in an Acceleo module to find out invalid modules, few of those detect logical problems (an OCL keyword used as the name of a template for example).

Experimenting is fun

This prototype of Acceleo parser in Scala can successfully parse all the valid Acceleo modules in the Acceleo parser unit tests and it detect at least one problem in all the invalid ones with only 217 lines of code. It would be completely irrelevant to compare both numbers like that but it illustrates how the support for parser in the Scala standard library can help to write very concise and efficient parsers. This Scala parser does not have any comment and it is missing the detection of about a third of all the errors detected by the original Acceleo parser, yet I believe that I can achieve the same level of quality for about 400 lines of code.

Talk is cheap. Show me the code. (Linus Torvalds)

The problem with writing parsers in Java is the lack of support for parsers in the Java standard library. On the contrary, in Scala, it is very easy to start writing a parser because Scala provides you with lot of basic implementations of parsers.

After spending some time on internet following some tutorials to master those implementations, I quickly realized that even if those implementations would allow me to do 80% of the work in no time, the remaining 20% would be a real challenge and I would have to create my own customs parsers. In Acceleo, text literal and static text in a template can contain pretty much all the delimiters and tokens that you can think of. Regular expressions and token based parsers cannot come even close to offer a valuable solution to parse pieces of text with such complex patterns.

So, in order to do handle this problem, I had to create custom parsers and for that there is only one concept to understand. Since Scala is object oriented and a functional language, methods are first class citizens just like object and they can be used as a parameter of an other method for example. With the help of closures you can access variables out of the scope of your method and write easily concise and efficient piece of code but there is one thing often forgotten. Since method are first class citizens, you can also extends them, so in the end, a parser is just a subclass of a function that takes an “Input” and returns a “ParserResult”.

abstract class Parser[+T] extends (Input => ParseResult[T]) {…}

To create your own custom parser, you just have to create a function that will returns a “Success” or a “Failure” object like in the following example.

I also discover that you cannot create a parser combinator that combines several parser combinators from other classes, a problem often ignored in most tutorials yet it can be bypassed in an elegant way by mixin traits.

I’ll need to work more on this prototype but I’m progressing very quickly since Scala and Java have lots of similar concepts. So stay tune for more information.

Acceleo Unit 0.1

It’s been a long time since I’ve worked on the unit testing framework for Acceleo. At that time, I had all the basis for the framework but I was missing some key elements and few hours of work to have a working prototype. Now that Acceleo 3.1 has been released, I had a bit more time to try and experiment and now I do have my working prototype.

First of all, it is still a prototype, as such it is still a bit raw and I have identify several new JUnit based annotations and some other utility operations that I should add to this framework in order to make it very easy for people to test their templates and queries. I also had to fork and override some key elements of the Acceleo engine in order to be able to access some elements of the OCL environment.

I would need new APIs in Acceleo in order to handle unit testing of queries. Since I didn’t want to have to touch any piece of code of Acceleo for this work, I didn’t create those APIs. Right now, I could easily evaluate any template containing a file block but with my fork of some Acceleo components I can now test all templates. As you can see in this screenshot of the unit test and the previous screenshot, I can test an Acceleo module compiled or not, in the workspace or in a plugin and with an EMF based model that also can be located pretty much anywhere too.

The method annotated ”modelElements” is a bit long, but it is because I am loading my UML model manually and then iterating on all the available model elements to select the first UML class. You could also use the method annotated “uriFragments” or the attribute annotated “uriFragment” to just give the uri fragments of the model elements that you want to test. So in the end, you can select by hand the model elements that will be used for the test or you can just give their uri and let Acceleo handle the rest. Finally it will even be available in stand alone without any dependencies with Eclipse.

I’ll have to discuss with the remaining members of the Acceleo dev’ team to determine how we could provide the necessary APIs for this framework but rest assured I have not forgotten this project and I know that some people out there are very interesting by it and I am planning on improving it a lot with some killer features. By the way, don’t forget to have a look at the new features coming with Acceleo 3.1 that has been released yesterday.

You can find more information on Acceleo on the Eclipse website, the Obeo Network or by following me on Twitter.

Using the Acceleo runtime in a Java project

Right now if you want to use the Acceleo runtime in a Java project you have to add the jars of the Acceleo runtime as external jars or even worst, you have to copy them in your Java project. In order to do that, you first have to find all the jars of the Acceleo runtime and their dependencies in your Eclipse installation folder. It is definitely not user friendly.

Looking for the Acceleo runtime dependencies ?

The Acceleo runtime will now be available directly from the Eclipse classpath configuration window.

From there you will be able to easily add the Acceleo runtime or even the Acceleo compiler as a dependency of your project.

This new feature will be available starting with Acceleo 3.1.0M7 along with the new syntax coloring preference menu and the OCL/Acceleo operations documentation in the Acceleo editor.

Small details always count.

I was fixing a small bug in the new Acceleo project wizard when I had to create a new Java class. I am creating classes all the time and I am using the new class wizard from the JDT everyday, yet once again an invisible feature of the JDT struck me.

Creating a Java class lasts less than a second, yet the new class wizard of the JDT features a small progress monitor at the bottom. It’s one of those features that change the perception of the response time of the interface by the user. And this kind of things can change everything between a dialog box that takes 1s without responding to complete its task which seems slow and a small progress bar that appears for 1s and by the time you realize that a task is going on, its already over. You don’t know for sure what happened but you know that it was fast.

And one of the main advantage of open source is that if you want to know how they did it, you just have to look at the code. And since, just like last time, I knew where to look for this feature, the project wizard in Acceleo 3.1M7 will feature that small progress monitor.

From an Acceleo code generator to an Eclipse plugin

Acceleo helps you to create, improve, deploy and maintain easily a code generator. While most of users know Acceleo as an Eclipse plugin used to create a code generators, they tend to forget that the Acceleo runtime can also be embedded in other tools. You can use the Java launcher created by Acceleo to launch the generation of your main templates from any kind of Java application and you can also deploy your generator as another Eclipse plugin very easily. Today I would like to focus this post on the improvements made by Laurent and I on the deployment of Acceleo generators as Eclipse plugins.

First of all, you will have to start by creating your code generator. While doing so, you can see that Acceleo will create several files that will ease the deployment of the generator like “build.properties” and “build.acceleo”. Those files contains all the necessary informations to build the Acceleo project.

When you generator is ready to be deployed, you can create its Eclipse feature. In order to do that, you just have to create a new feature project and then you have to initialize it with your Acceleo project.

Then you can create the update site for this feature by creating a new update site project that will be responsible of the deployment of the feature created earlier. When your update site is done, you just have to click on the build all button. This action will compile your Acceleo modules and prepare them for their deployment.

Finally you just have to make your update site available to your customers. Then, they can install your generator from your update site like any other Eclipse plug-in. When the generator is installed, it is integrated seamlessly in the Eclipse environment. Your customers won’t even realized that Acceleo is used.

Improved properties files processing in Acceleo 3.1

I’ve just finished working on the new system to handle properties files that will be available in the next milestone, so let’s see what’s coming. In Acceleo, you can create a “.properties” file to store some properties that will be used during your generation. You can see in the following screenshot how to use a properties file in the current version of Acceleo.

There are several steps to respect in order to use a properties file:

1. Change the “@generated” tag to “@notgenerated” in the javadoc of the method “getProperties()” in the Java class generated by Acceleo for your main module. Otherwise, any changes in the matching Acceleo module will override your modifications.
2. Add the path of your properties file in the list returned by “getProperties()”. Follow the same convention for the path of the file as in the screenshot.
3. Well, you have to create the properties file :)
4. Finally, you have to use your properties in your generator

The main advantage of the properties file is to allow the creator of the generator to configure some details of the generation without modifying the generator. You could for example, have a property to determine if you want to generate the author of a class in the javadoc or to change your naming conventions, etc.

You also have access to a “properties” text area in the launch configuration of Acceleo. In Acceleo 3.0, this text area should not be used, as its behavior is buggy.

In Acceleo 3.1, I’ve improved this system with the help of Goulwen the leader of the EEF project. You can now use “getProperties()” or the text area of the launch configuration without any distinctions to enter the path of a properties file. This path can still be a project relative path like in the previous screenshot but you can also use the absolute path of the properties file (“/home/…./default.properties”). If you want to use a properties file with another kind of path, you can now override “getPropertiesLoaderService(AcceleoService acceleoService)” and you can create an AbstractAcceleoPropertiesLoaderService.

The basic implementation of this class will look for the properties file in your project of at the location of the given path. You just have to implement “alternatePropertiesLoading(String filepath): Properties” to look for the properties file by yourself and then return the matching “java.util.Properties” object. With this, you will be able to look for properties file in a bundle or next to your models with an Acceleo UI project, etc. If you want to handle by yourself the whole process you can override “initializeService(List<String> propertiesFiles)”.

These changes will be available in Acceleo 3.1 M6, available next week.

Syntax highlighting in folded code for Acceleo

If there is one thing that I’ve learned while using Eclipse and creating Eclipse plugins it is that the devil is in the details. The development team of the JDT is one perfect example of this expression and I’ve seen countless of time in the code of the JDT dozens of lines of code written in order to handle a barely visible feature that seems just natural. You know, those small details that you don’t even consider as a feature, because they are just… there.

One of those features stroke me yesterday, you may never have realized it but in the JDT, when you are using the code folding mechanism (by collapsing a block of code) the Java syntax highlighting is available in the hover pop up of the folded code. Since I encountered a similar problem while working on the highlighting of the pop up of the Acceleo documentation I knew where I had to look for in order to achieve the same feature in Acceleo. And since I knew how I could do it, well… I had to it of course :)

As such, the syntax highlighting process in the code folding pop up will be available in Acceleo 3.1 M6 next week.

New serialization option and performance improvements in Acceleo 3.1

Acceleo 3.1 will feature a new option that will allow our user to chose their type of compilation. Right now, Acceleo modules are compiled as XMI resources and in the next release of Acceleo, you will have the ability to compile Acceleo modules as Binary resources.

For those who are not familiar with the serialization process of EMF models, let see how it will improve your live. With the XMI resource serialization, the Acceleo module is serialized in a XML file. In case of problems, you can open or edit it in a simple text editor and you can always check easily the result of the compilation. It is very helpful to debug any problem that has occurred during the compilation, yet it has one major problem: it produce large files.

On the contrary, the Binary resource serialization will produce an “.emtl” file (a compiled Acceleo module) way smaller than with the XMI serialization but you can’t easily read this file in a text editor.

Starting with Acceleo 3.1, all Acceleo modules will be serialized as Binary resources by default. Those who want to keep the old serialization system will be able to do so in the properties of their Acceleo project. With this new setting, Acceleo modules will be way smaller and a bit quicker to load. As an example, the Acceleo modules used by the project EEF are 5 times smaller with Binary Resources and the full compilation of the EEF modules is 10% to 15% faster. This new feature will be available with Acceleo 3.1 M6 next week.