avatar

Xtext and more

Xtext: Referencing Elements of one DSL from another DSL

August 7, 2012 · Christian Dietrich · Tags: Cross-Reference , Import , Inter-Language , Xtext

This is a blog post on Inter-Language-Cross-References in Xtext. Let us asume we have a DSL that contains Definitions. Now we want to create another DSL that references (uses) the Definitions defined in the the first DSL.

So let us start with the first DSL: We create a new Xtext Project

New Project Wizard First DSL

And here is the (for demonstration purpose oversimplyfied) Grammar

grammar org.xtext.example.definitions.Definitions with org.eclipse.xtext.common.Terminals
 
generate definitions "http://www.xtext.org/example/definitions/Definitions"
 
Model:
    definitions+=Definition*;
 
Definition:
    'define' name=ID;

We run GenerateDefinitions.mwe2 to generate the language. This is all for the first DSL.

Now let us create a project for the second DSL.

New Project Wizard Second DSL

To be able to reference the first DSL from the second we add a bundle dependency to the Manifest of the second

Dependencies

Here the resulting manifest

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: org.xtext.example.usages
Bundle-Vendor: My Company
Bundle-Version: 1.0.0.qualifier
Bundle-SymbolicName: org.xtext.example.usages; singleton:=true
Bundle-ActivationPolicy: lazy
Require-Bundle: org.eclipse.xtext;visibility:=reexport,
 org.eclipse.xtext.xbase;resolution:=optional;visibility:=reexport,
 org.eclipse.xtext.generator;resolution:=optional,
 org.apache.commons.logging;bundle-version="1.0.4";resolution:=optional,
 org.eclipse.emf.codegen.ecore;resolution:=optional,
 org.eclipse.emf.mwe.utils;resolution:=optional,
 org.eclipse.emf.mwe2.launch;resolution:=optional,
 org.xtext.example.definitions;bundle-version="1.0.0"
Import-Package: org.apache.log4j
Bundle-RequiredExecutionEnvironment: J2SE-1.5

Now we can create the Grammar from the Usage DSL

grammar org.xtext.example.usages.Usages with org.eclipse.xtext.common.Terminals
 
generate usages "http://www.xtext.org/example/usages/Usages"
 
import "http://www.xtext.org/example/definitions/Definitions" as def
 
Model:
    usages+=Usage*;
     
Usage:
    'use' definition=[def::Definition];

With "http://www.xtext.org/example/definitions/Definitions" as def we import the metamodel of our Define DSL to our Usage DSL. Thus the Type Definition is available and can be used to define the cross reference definition=[def::Definition]

To get the thing running we have to do some adjustments to the workflow of the Usages language.

module org.xtext.example.usages.GenerateUsages
 
import org.eclipse.emf.mwe.utils.*
import org.eclipse.xtext.generator.*
import org.eclipse.xtext.ui.generator.*
 
var grammarURI = "classpath:/org/xtext/example/usages/Usages.xtext"
var file.extensions = "use"
var projectName = "org.xtext.example.usages"
var runtimeProject = "../${projectName}"
 
Workflow {
    bean = StandaloneSetup {
        scanClassPath = true
        platformUri = "${runtimeProject}/..";
        registerGeneratedEPackage = "org.xtext.example.definitions.definitions.DefinitionsPackage"
        registerGenModelFile = "platform:/resource/org.xtext.example.definitions/src-gen/org/xtext/example/definitions/Definitions.genmodel"
    }
     
        ...
}

we generate our Usage DSL GenerateUsages.mwe2

finally we start a new runtime eclipse and give it a try

Result

Update Xtext 2.9.2+:

Here is how the workflow looks like with the new Xtext 2.9.x generator

module org.xtext.example.usages.GenerateUsages
 
import org.eclipse.xtext.xtext.generator.*
import org.eclipse.xtext.xtext.generator.model.project.*
 
var rootPath = ".."
 
Workflow {
     
    component = XtextGenerator {
        configuration = {
            project = StandardProjectConfig {
                baseName = "org.xtext.example.usages"
                rootPath = rootPath
                runtimeTest = {
                    enabled = true
                }
                eclipsePlugin = {
                    enabled = true
                }
                eclipsePluginTest = {
                    enabled = true
                }
                createEclipseMetaData = true
            }
            code = {
                encoding = "UTF-8"
                fileHeader = "/*\n * generated by Xtext \${version}\n */"
            }
        }
        language = StandardLanguage {
            name = "org.xtext.example.usages.Usages"
            referencedResource = "platform:/resource/org.xtext.example.definitions/model/generated/Definitions.genmodel"
            fileExtensions = "use"
 
            serializer = {
                generateStub = false
            }
            validator = {
            }
        }
    }
}

Unittesting Xtend Generators

May 8, 2012 · hristian Dietrich · Tags: Generator , Unit Test , Xtext

Xtext offers nice Support for Unit Tests. But how to test a Xtend based Generator? This blogpost describes a simple approach for such a Test.

So let us take Xtext’s Hello World grammar as Starting point

Model:
    greetings+=Greeting*;
     
Greeting:
    'Hello' name=ID '!';

And following simple Generator

package org.xtext.example.mydsl.generator
 
import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.xtext.generator.IGenerator
import org.xtext.example.mydsl.myDsl.Greeting
 
class MyDslGenerator implements IGenerator {
     
    override void doGenerate(Resource resource, IFileSystemAccess fsa) {
        for (g : resource.allContents.toIterable.filter(typeof(Greeting))) {
            fsa.generateFile(g.name+".java", 
            '''
            public class «g.name» {
                 
            }
            ''')
        }
    }
}

And here the Test

import org.junit.Test
import org.junit.runner.RunWith
import org.eclipse.xtext.junit4.XtextRunner
import org.eclipse.xtext.junit4.InjectWith
import org.xtext.example.mydsl.MyDslInjectorProvider
import org.eclipse.xtext.generator.IGenerator
import com.google.inject.Inject
import org.eclipse.xtext.junit4.util.ParseHelper
import org.xtext.example.mydsl.myDsl.Model
import org.eclipse.xtext.generator.InMemoryFileSystemAccess
 
import static org.junit.Assert.*
import org.eclipse.xtext.generator.IFileSystemAccess
 
@RunWith(typeof(XtextRunner))
@InjectWith(typeof(MyDslInjectorProvider))
class GeneratorTest {
     
    @Inject IGenerator underTest
    @Inject ParseHelper<Model> parseHelper 
     
    @Test
    def test() {
        val model = parseHelper.parse('''
        Hello Alice!
        Hello Bob!
        ''')
        val fsa = new InMemoryFileSystemAccess()
        underTest.doGenerate(model.eResource, fsa)
        println(fsa.files)
        assertEquals(2,fsa.files.size)
        assertTrue(fsa.files.containsKey(IFileSystemAccess::DEFAULT_OUTPUT+"Alice.java"))
        assertEquals(
            '''
            public class Alice {
                 
            }
            '''.toString, fsa.files.get(IFileSystemAccess::DEFAULT_OUTPUT+"Alice.java").toString
        )
        assertTrue(fsa.files.containsKey(IFileSystemAccess::DEFAULT_OUTPUT+"Bob.java"))
        assertEquals(
            '''
            public class Bob {
                 
            }
            '''.toString, fsa.files.get(IFileSystemAccess::DEFAULT_OUTPUT+"Bob.java").toString)
         
    }
     
}

But how does that work?

Xtext offers a specific org.junit.runner.Runner. For Junit4 it is org.junit.runner.Runner. This Runner allows in combination with a org.eclipse.xtext.junit4.IInjectorProvide language specific injections within the test. Since we have fragment = junit.Junit4Fragment {} in our workflow Xtext already Generated the Class org.xtext.example.mydsl.MyDslInjectorProvider. If we would not use Xtext at all we would have to create such a InjectorProvider manually.

To wire these things up we annotate your Test with @RunWith(typeof(XtextRunner)) and @InjectWith(typeof(MyDslInjectorProvider))

Now we can write our Test. This Basically consists of 3 steps

  1. read a model
  2. call the Generator
  3. Capture the Result

We solve Step (1) using Xtext’s org.eclipse.xtext.junit4.util.ParseHelper and Step (3) by using a special kind of IFileSystemAccess that keeps the files InMemory and does not write them to the disk.

I hope this gives you a start writing you Xtext/Xtend Generator Tests.

Xtext Content Assist Auto Activation

September 19, 2011 · Christian Dietrich · Tags: Auto Activation , Content Assist , Xtext

Xtext offers nice Content Assist facilities. JDT offers a nice additional feature: Content assist is autoactivated if a certain character (.) is typed. To activate this feature in Xtext simply customize your UiModule

public class MyDslUiModule extends org.xtext.example.mydsl.ui.AbstractMyDslUiModule {
    public MyDslUiModule(AbstractUIPlugin plugin) {
        super(plugin);
    }
     
    @Override
    public void configure(Binder binder) {
        super.configure(binder);
    	binder.bind(String.class)
			.annotatedWith(com.google.inject.name.Names.named(
			(XtextContentAssistProcessor.COMPLETION_AUTO_ACTIVATION_CHARS)))
			.toInstance(".,:");
    }
}

In this case content assist is autoactivated on . , and :

Xtend2 Code Generators with Non-Xtext Models

July 29, 2011 · Christian Dietrich · Tags: EMF , Xtend , Xtext

In this blog post i want to show a simple example of how to use Xtend2 to generate code from Non-Xtext but EMF-based model.

Xtend with EMF

Having a simple EMF Model i’ve created the genmodel + Model + Edit + Editor code. Using the Editor i’ve created a bunch of .sample files and now want to generate code using Xtend2

Xtend EMF Models

Xtend comes with an IGenerator interface that i implement in my SampleGenerator Xtend file

package sample
 
import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.emf.ecore.EObject
 
class SampleGenerator implements IGenerator {
 
    override void doGenerate(Resource resource, IFileSystemAccess fsa) {
        for (EObject o : resource.contents) {
            o.compile(fsa)
        }
    }
 
    def dispatch void compile(Model m, IFileSystemAccess fsa) {
        for (e : m.elements) {
            e.compile(fsa)
        }
    }
 
    def compile(Element e, IFileSystemAccess fsa) {
        fsa.generateFile(e.name+".txt", '''
        this is element «e.name»
        ''')
    }
 
    def dispatch void compile(EObject m, IFileSystemAccess fsa) { }
 
}

The last step we need is a workflow that reads the model files and invokes the generator

First we need to create some java classes that exposes our .sample to the reader (resourceseriveprovider) and do some Guice Binding Stuff (Generator / ResourceSet ….)

package sample;
 
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.emf.ecore.resource.impl.ResourceSetImpl;
import org.eclipse.xtext.generator.IGenerator;
import org.eclipse.xtext.resource.generic.AbstractGenericResourceRuntimeModule;
 
public class SampleGeneratorModule extends AbstractGenericResourceRuntimeModule {
 
    @Override
    protected String getLanguageName() {
        return "sample.presentation.SampleEditorID";
    }
 
    @Override
    protected String getFileExtensions() {
        return "sample";
    }
 
    public Class<? extends IGenerator> bindIGenerator() {
        return SampleGenerator.class;
    }
 
    public Class<? extends ResourceSet> bindResourceSet() {
        return ResourceSetImpl.class;
    }
 
}

package sample;
 
import org.eclipse.xtext.ISetup;
 
import com.google.inject.Guice;
import com.google.inject.Injector;
 
public class SampleGeneratorSetup implements ISetup {
 
    @Override
    public Injector createInjectorAndDoEMFRegistration() {
        return Guice.createInjector(new SampleGeneratorModule());
    }
 
}

package sample;
 
import org.eclipse.xtext.resource.generic.AbstractGenericResourceSupport;
 
import com.google.inject.Module;
 
public class SampleGeneratorSupport extends AbstractGenericResourceSupport {
 
    @Override
    protected Module createGuiceModule() {
        return new SampleGeneratorModule();
    }
 
}

finally we wire this together in the workflow file

module sample.SampleGenerator
 
import org.eclipse.emf.mwe.utils.*
 
var targetDir = "src-gen"
var modelPath = "model"
 
Workflow {
 
    bean = StandaloneSetup {
        registerGeneratedEPackage = "sample.SamplePackage"
    }
 
    component = DirectoryCleaner {
        directory = targetDir
    }
 
    component = sample.SampleGeneratorSupport {}
 
    component = org.eclipse.xtext.mwe.Reader {
        path = modelPath
        register = sample.SampleGeneratorSetup {}
        loadResource = {
            slot = "model"
        }
    }
 
    component = org.eclipse.xtext.generator.GeneratorComponent {
        register = sample.SampleGeneratorSetup {}
        slot = 'model'
        outlet = {
            path = targetDir
        }
    }
}

running the workflow we get nice files generated

Xtend EMF Models + Generated

Customizing Xtext Metamodel Inference using Xtend2

July 22, 2011 · Christian Dietrich · Tags: Inference , Metamodel , PostProcessor , Xtend , Xtext

Xtext has nice metamodel inference capabilities. But sometimes you have to do some customizations to the generated ecore metamodel, e.g. adding a derived operation. You have basically two options: (1) move to a manually maintained metamodel (2) use Customized Post Processing as described here http://www.eclipse.org/Xtext/documentation/2_0_0/020-grammar-language.php#customPostProcessing

The second possibility uses good old Xpand/Xtend1 extensions to do the postprocessing. But what if i want to use Xtend2 for that? A very simple solution i’d like to show in this post.

So lets start with the gramar

grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals
 
generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"
 
Model:
    persons+=Person*
;
 
Person:
    "person" firstname=STRING lastname=STRING
;

We now want to add a getFullname Operation to our person. Xtext offers the Interface IXtext2EcorePostProcessor for the postprocessing. So we write a Xtend class for that

package org.xtext.example.mydsl
 
import org.eclipse.xtext.xtext.ecoreInference.IXtext2EcorePostProcessor
import org.eclipse.xtext.GeneratedMetamodel
import org.eclipse.emf.ecore.EPackage
import org.eclipse.emf.ecore.EClassifier
import org.eclipse.emf.ecore.EClass
import org.eclipse.emf.ecore.EcoreFactory
import org.eclipse.emf.ecore.EcorePackage
import org.eclipse.emf.ecore.EcorePackage.Literals
import org.eclipse.emf.codegen.ecore.genmodel.GenModelPackage
import org.eclipse.emf.common.util.BasicEMap$Entry
import org.eclipse.emf.ecore.impl.EStringToStringMapEntryImpl
 
class MyXtext2EcorePostProcessor implements IXtext2EcorePostProcessor {
     
    override void process(GeneratedMetamodel metamodel) {
        metamodel.EPackage.process
    }
     
    def process(EPackage p) {
        for (c : p.EClassifiers.filter(typeof(EClass))) {
            if (c.name == "Person") {
                c.handle
            }
        }
    }
     
    def handle (EClass c) {
        val op = EcoreFactory::eINSTANCE.createEOperation
        op.name = "getFullName"
        op.EType = EcorePackage::eINSTANCE.EString
        val body = EcoreFactory::eINSTANCE.createEAnnotation
        body.source = GenModelPackage::eNS_URI
        val map = EcoreFactory::eINSTANCE.create(EcorePackage::eINSTANCE.getEStringToStringMapEntry()) as BasicEMap$Entry<String,String>
        map.key = "body"
        map.value = "return getFirstname() + \" \" + getLastname();"
        body.details.add(map)
        op.EAnnotations += body
        c.EOperations += op
    }
     
}

The last Problem left is how to make the Generator use this class. Xtext does not offer a explicit place to change the IXtext2EcorePostProcessor. Its default Implementation is bound in the XtextRuntimeModule, that is instantiated in the org.eclipse.xtext.generator.Generator class. so we have to subclass the Generator to get it changed

package org.xtext.example.mydsl;
 
import org.eclipse.xtext.XtextRuntimeModule;
import org.eclipse.xtext.XtextStandaloneSetup;
import org.eclipse.xtext.generator.Generator;
import org.eclipse.xtext.xtext.ecoreInference.IXtext2EcorePostProcessor;
 
import com.google.inject.Guice;
import com.google.inject.Injector;
 
@SuppressWarnings("restriction")
public class ExtendedGenerator extends Generator {
     
    public ExtendedGenerator() {
        new XtextStandaloneSetup() {
            @Override
            public Injector createInjector() {
                return Guice.createInjector(new XtextRuntimeModule() {
                    @Override
                    public Class<? extends IXtext2EcorePostProcessor> bindIXtext2EcorePostProcessor() {
                        return MyXtext2EcorePostProcessor.class;
                    }
                });
            }
        }.createInjectorAndDoEMFRegistration();
    }
 
}

finally we use the in the Generator Workflow

Workflow {
    bean = StandaloneSetup {
        scanClassPath = true
        platformUri = "${runtimeProject}/.."
    }
 
    component = DirectoryCleaner {
        directory = "${runtimeProject}/src-gen"
    }
 
    component = DirectoryCleaner {
        directory = "${runtimeProject}.ui/src-gen"
    }
 
    component = ExtendedGenerator {
        pathRtProject = runtimeProject
        pathUiProject = "${runtimeProject}.ui"
        pathTestProject = "${runtimeProject}.tests"
        projectNameRt = projectName
        projectNameUi = "${projectName}.ui"
        language = {
....

as a result our person has the getFullname Operation

public interface Person extends EObject
{
 
  String getFirstname();
 
 
  void setFirstname(String value);
 
 
  String getLastname();
 
 
  void setLastname(String value);
 
 
  String getFullName();
 
} // Person

public class PersonImpl extends MinimalEObjectImpl.Container implements Person
{
  
  /**
   * <!-- begin-user-doc -->
   * <!-- end-user-doc -->
   * @generated
   */
  public String getFullName()
  {
    return getFirstname() + " " + getLastname();
  }
 
 
 
} //PersonImpl