benerator file format

The benerator configuration file is XML based. An XML schema is provided. The document root is a setup element:

<?xml version="1.0" encoding="iso-8859-1"?>
<setup xmlns="http://databene.org/benerator-0.7.0.xsd"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://databene.org/benerator-0.7.0.xsd benerator-0.7.0.xsd">
<setup>
    <!-- content here -->
</setup>

benerator files should end with the suffix .ben.xml.

properties

You can define global properties:

        <property name="my_name" value="Volker" />

or import several of them from a properties file:

        <include uri="my.properties" />

javabeans and the context

You can instantiate JavaBeans by an intuitive syntax like this:

  <bean id="db" class="com.my.SpecialBean">
      <property name="user" value="benerator"/>
      <property name="password" value="benerator"/>
  </bean>

The class attribute denotes which JavaBean class to instantiate (by the default constructor). The enclosed property tags cause the JavaBean's properties to be set to appropriate values. Benerator converts common types automatically. If not, you may define a custom ConverterManager setup (see databene-commons). Date and time formatting is supported according to ISO 8601 Conventions.

Objects are made available by exposing them in a context. The id attribute defines the name with which an object can be found, e.g. for a 'source' or 'ref' attribute of another element's setup.

So the example above creates an instance of a DBSystem JavaBean class, setting its properties to values for connecting a database. The object is retrievable by the context with the id 'db'.

Note: The class DBSystem implements the interface 'System' which provides (among other features) meta information about the entities (tables) contained in the database.

You can create references to other objects declared before by a 'ref'-attribute in the bean declaration. The following example shows this for a task setup, but this can be applied to beans and consumers as well.

Note: You may implement the System interface for connecting to other system types like SAP or Siebel systems.

JavaBeans may refer each other and may have collection or attribute properties as shown in the following example:

databases can be defined using a <database> element:

tasks

Besides the general 'bean' elements, other elements may be created by using element name and additional attributes as processing information.

The example above tells to create a JavaBean of class 'RunSqlScriptTask' with a uri 'shop/create_tables.mysql.sql' and its property 'db' refering the JavaBean "db" in the context. Finally it is executed.

You may define custom tasks to suit your needs, e.g. for performing health checks, by implementing the interface 'org.databene.task.Task'. By its interface a Tasks demarks if it is thread-safe or at least parallelizable.

The element run-task also supports the attributes

• count: the total number of times the Task is executed (defaults to 1)
• pagesize: the number of invocations to execute 'en bloque' (defaults to 1)
• threads: the number of threads with which to execute the Task (defaults to 1)

importing entities

Entities can be imported from 'system's, files or other generators. A typical application is to (re)use a DBUnit setup file from your (hopefully existing ;-) unit tests:

For importing DbUnit files, follow the naming conventions using the suffix .dbunit.xml.

Each created entity is forwarded to one or more consumers, which usually will persist objects in a file or system, but might also be used to post-process created entities. The specified object needs to implement the Consumer or the system interface. When specifying a system here, it will be used to store the entities. File exporters (for CSV and Flat Files) implement the Consumer interface.

custom importers

New import formats can be supported by implementing the EntitySource interface with a JavaBean implementation, instantiating it as bean and refering it by its id with a 'source' attribute, e.g.

chaining generators

Generators may be chained, composed, or reused in different contexts. You can do so by instantiating a generator as JavaBean and referring it in properties of other JavaBean-instantiated generators or specifying it as 'source' attribute like an importer.

creating random entities

Entities can be generated without any input files: Benerator provides a rich set of Generator implementations. When using generate, the registered systems (e.g. the database) are queried for meta data. Benerator interprets the meta data and automatically sets up generators that match the systems' constraints, lik column length, referenced entities and more. By default, associations are treated as many-to-one associations.

    <!-- create products of random attribs & category -->
    <generate name="db_product" count="1000" pagesize="100">
        <consumer ref="db"/>
    </generate>

Entities are generated as long as each attribute generator is available and limited by the number specified in the 'count' attribute. The 'pagesize' defines the number of creations after which a flush() is applied to all consumers (for a database system this is mapped to a commit).

nesting entities

Entities can form composition structures, which are generated best by recursive generate structures.

TODO: example

exporting generated data to data files

You will need to reuse some of the generated data for setting up (load) test clients. You can simply export data by an appropriate consumer:

    <!-- create products of random attribs & category -->
    <generate name="db_product" count="1000" pagesize="100">
        <consumer ref="db"/>
        <consumer class="org.databene.platform.fixedwidth.FixedWidthEntityExporter">
            <property name="uri" value="products.flat"/>
            <property name="properties" value="ean_code[13],name[30l],price[10r0]"/>
        </consumer>
    </generate>

imposing one-field business constraints

Simple constraints, e.g. formats can be assured by defining an appropriate Generator or regular expression, e.g.

    <!-- create products of random attribs & category -->
    <generate name="db_product" count="1000" pagesize="100">
        <attribute name="ean_code" generator="org.databene.domain.product.EANGenerator"/>
        <attribute name="name" pattern="[A-Z][A-Z]{5,12}"/>
        <consumer ref="db"/>
    </generate>

imposing multi-field-constraints

For supporting multi-field-constraints, you can provide a Generator (with a variable element) that creates entities, JavaBeans or Maps. This may be e.g. a random generator or an importing generator. On each generation run, an instance is generated and made available to the other sub generators. They can use the entity or sub elements by a source path attribute:

    <generate name="db_customer">
        <variable name="person" generator="org.databene.domain.person.PersonGenerator" dataset="DE"/>
        <attribute name="salutation" source="person.salutation"/>
        <attribute name="first_name" source="person.givenName"/>
        <attribute name="last_name" source="person.familyName"/>
        <consumer ref="db"/>
    </generate>

The source path may be composed of property names, map keys and entity features, separated by a dot.

data types

The following data types are supported:

querying information from a system

Arbitrary information may be queried from a system by a 'selector' attribute, which is system-dependent. For a database SQL is used:

    <generate name="db_order" count="30" pagesize="100">
        <attribute name="id" mode="ignored"/>
        <attribute name="customer_id" source="db" selector="select id from db_customer" cyclic="true"/>
        <consumer ref="db"/>
    </generate>

The result set of a selector might be quite large, so different strategies (for wrapping any other generator's output) are supported:

• distribution: Maps to the name of a Sequence or WeightFunction class. For this, the complete result set is loaded into ram. A Sequence should not be applied to result sets of more than 100.000 elements, a WeightFunction should be restricted to at most 10.000 elements.
• proxy="skip" or proxy="repeat" for iterating sequentially through the set. 'proxy-param1' and 'proxy-param2' may be used to specify minimum and maximum of repetitions or skipped elements. If cyclic="true", the result set will be re-iterated from the beginning when it has reached the end.

      <generate name="db_order_item" count="100" pagesize="100">
          <attribute name="id" mode="ignored"/>
          <attribute name="number_of_items" min="1" max="27" distribution="cumulated"/>
          <attribute name="order_id" source="db" selector="select id from db_order" cyclic="true"/>
          <attribute name="product_id" source="db" selector="select ean_code from db_product" distribution="random"/>
          <consumer ref="db"/>
      </generate>

    selector="{select ean_code from db_product where country='${country}'}"

The script is resolved immediately before the first generation and then reused.

If you need dynamic queries, that are re-evaluated, you can specify them with double brackets:

selector="{{select ean_code from db_product where country='${shop.country}'}}"

Example:

    <generate name="shop" count="10">
        <attribute name="country" values="DE,AT,CH"/>
        <generate name="product" count="100" consumer="db">
            <attribute name="ean_code" source="db" selector="selector="{{select ean_code from db_product where country='${shop.country}'}}"/>
        </generate>
    </generate> 

entity definition

id definition

attribute definition

reference definition

Scripting

Scripts are supported in

• benerator setup files
• properties files
• DbUnit XML files
• CSV files
• Flat files

  A script is denoted by curly braces, e.g. '{Hi, I am ${my_name}}'. This syntax will use the default script engine for rendering the text as, e.g. 'Hi, I am Volker'.

  The default script engine is set by the property benerator.defaultScript .

  If you need to support different script engines (e.g. while combining files from different sources), you can differ them by prepending the scripting engine id, e.g. '{ftl:Hi, I am ${my_name}}' or '{Vel:Hi, I am ${my_name}}'

  Scripts in the benerator setup are evaluated while parsing. If you need to dynamically generate script text at runtime, use a attribute.script field:

   <attribute name="total_price" script="{${(product[1] * db_order_item.number_of_items)?c}}" />

  With scripts you can access

• environment variables
• JVM parameters
• any JavaBean globally declared in the benerator setup
• the last generated entity of each type
• variable values

  Variable names in scripting may not contain points - a point always implies navigation, e.g. person.familyName navigates from the person object to the familyName attribute/property/key.

staging

Combining scripting and property files, you get a staging mechanism, which is demonstarted in the shop demo. Check the file demo/shop/shop.ben.xml in your benerator installation. It uses staging for populating all seven supported databases with the same benerator setup file, moving database specific code to small properties files.

When invoking benerator with a -Dstage=development JVM parameter, you can make your import

include uri="{demo/shop/shop.${stage}.properties}" /

template support

You can use DbUnit import files for replicating entity graph structures many times on each generated object. Say, for each customer in a tested online shop, a default order structure should be created. You would then define the order structure in a DbUnit file

 <dataset>
    <db_order_item order_id="{${db_order.id}}" number_of_items="2" product_ean_code="8076800195057" total_price="2.40" />
    <db_order_item order_id="{${db_order.id}}" number_of_items="1" product_ean_code="8006550301040" total_price="8.70" />
 </dataset>

and then create an order for each customer that imports its sub structure from the DbUnit file:

        <generate name="db_order" consumer="db">
                <id name="id" generator="IncrementalIdGenerator" />
                <attribute name="customer_id" source="db" selector="select id from db_customer" />
                <iterate name="db_order_item" source="demo/shop/default_order.dbunit.xml" consumer="db">
                        <id name="id" generator="IncrementalIdGenerator" />
                </iterate>
        </generate>

Of course, you have to care for appropriate ids yourself.