3.2. Implementing the Framework 3.2.1. Caching API Translators may contribute cache entries to the result set cache by the use of the CacheDirective object. Translators wishing to participate in caching should return a CacheDirective from the ExecutionFactory.getCacheDirective method, which is called prior to execution. The commands passed to getCacheDirective will have already been vetted to ensure that the results are eligible for caching. For example update commands or commands with pushed dependent sets will not be eligible for caching. If the translator returns null for the CacheDirective , which is the default implementation, the engine will not cache the translator results beyond the current command. It is up to your custom translator or custom delegating translator to implement your desired caching policy. Note In special circumstances where the translator has performed its own caching, it can indicate to the engine that the results should not be cached or reused by setting the Scope to Scope.NONE . The returned CacheDirective will be set on the ExecutionContext and is available via the ExecutionContext.getCacheDirective() method. Having ExecutionFactory.getCacheDirective called prior to execution allows the translator to potentially be selective about which results to even attempt to cache. Since there is a resource overhead with creating and storing the cached results it may not be desirable to attempt to cache all results if it is possible to return large results that have a low usage factor. If you are unsure about whether to cache a particular command result you may return an initial CacheDirective then change the Scope to Scope.NONE at any time prior to the final cache entry being created and the engine will give up creating the entry and release its resources. Note If you plan on modifying the CacheDirective during execution, return a new instance from the ExecutionFactory.getCacheDirective call, rather than returning a shared instance. The CacheDirective readAll Boolean field is used to control whether the entire result should be read if not all of the results were consumed by the engine. If readAll is false then any partial usage of the result will not result in it being added as a cache entry. Partial use is determined after any implicit or explicit limit has been applied. The other fields on the CacheDirective object map to the cache hint options . See the table below for the default values for all options. option default scope Session ttl rs cache ttl readAll true updatable true prefersMemory false 3.2.2. Command Language 3.2.2.1. Language Red Hat JBoss Data Virtualization sends commands to your Translator in object form. These classes are all defined in the org.teiid.language package. These objects can be combined to represent any command sent to the Translator. However, it is possible to specify that your Translator can only accept certain kinds of constructs via the capabilities defined on the ExecutionFactory class. Refer to the section on translator capabilities for more information. The language objects all extend from the LanguageObject interface. Language objects should be thought of as a tree where each node is a language object that has zero or more child language objects of types that are dependent on the current node. All commands sent to your Translator are in the form of these language trees, where the root of the tree is a subclass of Command . Command has several sub-interfaces, namely: QueryExpression Insert Update Delete BatchedUpdates Call Important components of these commands are expressions, criteria, and joins, which are examined in closer detail below. For more on the classes and interfaces described here, refer to the Red Hat JBoss Data Virtualization Javadoc. 3.2.2.2. Expressions An expression represents a single value in context, although in some cases that value may change as the query is evaluated. For example, a literal value, such as 5 represents an integer value. A column reference such as "table.EmployeeName" represents a column in a data source and may take on many values while the command is being evaluated. Expression - base expression interface ColumnReference - represents an column in the data source Literal - represents a literal scalar value, but may also be multi-valued in the case of bulk updates. Function - represents a scalar function with parameters that are also Expressions AggregateFunction - represents an aggregate function which holds a single expression WindowFunction - represents a window function which holds an AggregateFunction (which is also used to represent analytical functions) and a WindowSpecification ScalarSubquery - represents a subquery that returns a single value SearchedCase, SearchedWhenClause - represents a searched CASE expression. The searched CASE expression evaluates the criteria in WHEN clauses until one of them evaluates to TRUE, then evaluates the associated THEN clause. Array - represents an array of expressions, currently only used by the engine in multi-attribute dependent joins - see the supportsArrayType capability. 3.2.2.3. Condition A criteria is a combination of expressions and operators that evaluates to true, false, or unknown. Criteria are most commonly used in the WHERE or HAVING clauses. Condition - the base criteria interface Not - used to NOT another criteria AndOr - used to combine other criteria via AND or OR SubqueryComparison - represents a comparison criteria with a subquery including a quantifier such as SOME or ALL Comparison - represents a comparison criteria with =, >, <, etc. BaseInCondition - base class for an IN criteria In - represents an IN criteria that has a set of expressions for values SubqueryIn - represents an IN criteria that uses a subquery to produce the value set IsNull - represents an IS NULL criteria Exists represents an EXISTS criteria that determines whether a subquery will return any values Like - represents a LIKE/SIMILAR TO/LIKE_REGEX criteria that compares string values 3.2.2.4. The FROM Clause The FROM clause contains a list of TableReference 's. NamedTable - represents a single Table Join - has a left and right TableReference and information on the join between the items DerivedTable - represents a table defined by an inline QueryExpression A list of TableReference are used by default, in the pushdown query when no outer joins are used. If an outer join is used anywhere in the join tree, there will be a tree of Join s with a single root. This latter form is the ANSI-preferred style. If you wish all pushdown queries containing joins to be in ANSI style have the capability "useAnsiJoin" return true. Refer to the section on command form for more information. 3.2.2.5. QueryExpression Structure QueryExpression is the base for both SELECT queries and set queries. It may optionally take an OrderBy (representing a SQL ORDER BY clause) and a Limit (represent a SQL LIMIT clause) or a With (represents a SQL WITH clause). 3.2.2.6. Select Structure Each QueryExpression can be a Select describing the expressions (typically elements) being selected and a TableReference specifying the table or tables being selected from, along with any join information. The Select may optionally also supply a Condition (representing a SQL WHERE clause), a GroupBy (representing a SQL GROUP BY clause), a Condition (representing a SQL HAVING clause). 3.2.2.7. SetQuery Structure A QueryExpression can also be a SetQuery that represents the SQL set operations (UNION, INTERSECT, EXCEPT) on two QueryExpression s. The all flag may be set to indicate UNION ALL (currently INTERSECT and EXCEPT ALL are not supported). 3.2.2.8. With Structure A With clause contains named QueryExpressions held by WithItem s that can be referenced as tables in the main QueryExpression . 3.2.2.9. Insert Structure Each Insert will have a single NamedTable specifying the table being inserted into. It will also has a list of ColumnReference specifying the columns of the NamedTable that are being inserted into. It also has InsertValueSource , which will be a list of Expressions ( ExpressionValueSource ), or a QueryExpression . 3.2.2.10. Update Structure Each Update will have a single NamedTable specifying the table being updated and list of SetClause entries that specify ColumnReference and Expression pairs for the update. The Update may optionally provide a criteria Condition specifying which rows should be updated. 3.2.2.11. Delete Structure Each Delete will have a single NamedTable specifying the table being deleted from. It may also optionally have a criteria specifying which rows should be deleted. 3.2.2.12. Call Structure Each Call has zero or more Argument objects. The Argument objects describe the input parameters, the output result set, and the output parameters. 3.2.2.13. BatchedUpdates Structure Each BatchedUpdates has a list of Command objects (which must be either Insert , Update or Delete ) that compose the batch. 3.2.2.14. The Type Facility The Translator API contains an interface TypeFacility that defines data types and provides value translation facilities. This interface can be obtained from calling the ExecutionFactory.getTypeFacility() method. The TypeFacility interface has methods that support data type transformation and detection of appropriate runtime or JDBC types. The TypeFacility.RUNTIME_TYPES and TypeFacility.RUNTIME_NAMES interfaces defines constants for all Red Hat JBoss Data Virtualization runtime data types. All Expression instances define a data type based on this set of types. These constants are often needed in understanding or creating language interfaces. 3.2.2.15. Language Manipulation In Translators that support a richer set of capabilities, there is often a need to manipulate or create language interfaces with a similar syntax to those being translated to. This is often the case when translating to a language comparable to SQL. Some utilities are provided for this purpose. Similar to the TypeFacility, you can call getLanguageFactory() method on the ExecutionFactory to get a reference to the LanguageFactory instance for your translator. This interface is a factory that can be used to create new instances of all the concrete language interface objects. Some helpful utilities for working with Condition objects are provided in the LanguageUtil class. This class has methods to combine Condition with AND or to break a Condition apart based on AND operators. These utilities are helpful for breaking apart a criteria into individual filters that your translator can implement. 3.2.2.16. Runtime Metadata Red Hat JBoss Data Virtualization uses a library of metadata, known as runtime metadata for each virtual database (VDB) that is deployed. The runtime metadata is a subset of the metadata defined by the models contributing to your VDB. While building your VDB in the Designer, you can define what called an Extension Model , that defines any number of arbitrary properties on a model and its objects. At runtime, using the runtime metadata interface, you can use properties that were defined at design time to define execution behavior. Translator gets access to the RuntimeMetadata interface at the time of Excecution creation. Translators can access runtime metadata by using the interfaces defined in org.teiid.metadata package. This package defines API representing a Schema, Table, Columns and Procedures, and ways to navigate these objects. 3.2.2.17. Metadata Objects All the language objects extend AbstractMetadataRecord class Column - returns Column metadata record Table - returns a Table metadata record Procedure - returns a Procedure metadata record ProcedureParameter - returns a Procedure Parameter metadata record Once a metadata record has been obtained, it is possible to use its metadata about that object or to find other related metadata. 3.2.2.18. Access to Runtime Metadata The RuntimeMetadata interface is passed in for the creation of an "Execution". See "createExecution" method on the "ExecutionFactory" class. It provides the ability to look up metadata records based on their fully qualified names in the VDB. Example 3.1. Obtaining Metadata Properties The process of getting a Table's properties is sometimes needed for translator development. For example to get the "NameInSource" property or all extension properties: //getting the Table metadata from an Table is straight-forward Table table = runtimeMetadata.getTable("table-name"); String contextName = table.getNameInSource(); //The props will contain extension properties Map<String, String> props = table.getProperties(); 3.2.2.19. Visitor Framework The API provides a language visitor framework in the org.teiid.language.visitor package. The framework provides utilities useful in navigating and extracting information from trees of language objects. The visitor framework is a variant of the Visitor design pattern, which is documented in several popular design pattern references. The visitor pattern encompasses two primary operations: traversing the nodes of a graph (also known as iteration) and performing some action at each node of the graph. In this case, the nodes are language interface objects and the graph is really a tree rooted at some node. The provided framework allows for customization of both aspects of visiting. The base AbstractLanguageVisitor class defines the visit methods for all leaf language interfaces that can exist in the tree. The LanguageObject interface defines an acceptVisitor() method. This method will call back on the visit method of the visitor to complete the contract. A base class with empty visit methods is provided as AbstractLanguageVisitor. The AbstractLanguageVisitor is a visitor shell - it performs no actions when visiting nodes and does not provide any iteration. The HierarchyVisitor provides the basic code for walking a language object tree. The HierarchyVisitor performs no action as it walks the tree - it encapsulates the knowledge of how to walk it. If your translator wants to provide a custom iteration that walks the objects in a special order (to exclude nodes, include nodes multiple times, conditionally include nodes, and so forth) then you must either extend HierarchyVisitor or build your own iteration visitor. In general, that is not necessary. The DelegatingHierarchyVisitor is a special subclass of the HierarchyVisitor that provides the ability to perform a different visitor's processing before and after iteration. This allows users of this class to implement either pre- or post-order processing based on the HierarchyVisitor. Two helper methods are provided on DelegatingHierarchyVisitor to aid in executing pre- and post-order visitors. 3.2.2.20. Provided Visitors The SQLStringVisitor is a special visitor that can traverse a tree of language interfaces and output the equivalent Red Hat JBoss Data Virtualization SQL. This visitor can be used to print language objects for debugging and logging. The SQLStringVisitor does not use the HierarchyVisitor described in the last section; it provides both iteration and processing type functionality in a single custom visitor. The CollectorVisitor is a handy utility to collect all language objects of a certain type in a tree. Some additional helper methods exist to do common tasks such as retrieving all elements in a tree, retrieving all groups in a tree, and so on. 3.2.2.21. Writing a Visitor Writing your own visitor can be quite easy if you use the provided facilities. If the normal method of iterating the language tree is sufficient, then follow these steps: Create a subclass of AbstractLanguageVisitor. Override any visit methods needed for your processing. For instance, if you wanted to count the number of column references in the tree, you need only override the visit(ColumnReference) method. Collect any state in local variables and provide accessor methods for that state. Decide whether to use pre-order or post-order iteration. Note that visitation order is based upon syntax ordering of SQL clauses - not processing order. Write code to execute your visitor using the utility methods on DelegatingHierarchyVisitor: // Get object tree LanguageObject objectTree = ... // Create your visitor initialize as necessary MyVisitor visitor = new MyVisitor(); // Call the visitor using pre-order visitation DelegatingHierarchyVisitor.preOrderVisit(visitor, objectTree); // Retrieve state collected while visiting int count = visitor.getCount(); 3.2.3. Connections to Source The extended "ExecutionFactory" must implement the getConnection() method to allow the Connector Manager to obtain a connection. Once the Connector Manager has obtained a connection, it will use that connection only for the lifetime of the request. When the request has completed, the closeConnection() method called on the "ExecutionFactory". You must also override this method to properly close the connection. In cases (such as when a connection is stateful and expensive to create), connections should be pooled. If the resource adapter is JEE JCA connector based, then pooling is automatically provided by the JBoss AS container. If your resource adapter does not implement the JEE JCA, then connection pooling semantics are left to the user to define on their own. 3.2.4. Dependent Join Pushdown Dependent joins are a technique used in federation to reduce the cost of cross source joins. Join values from one side of a join are made available to the other side which reduces the number of tuples needed to preform the join. Translators may indicate support for dependent join pushdown via the supportsDependentJoin capability. The handling of pushdown dependent join queries can be quite complicated. The ordering (if present) and all of the non-dependent criteria constructs on the pushdown command must be honored, but if needed the dependent criteria, which will be a Comparison with a Parameter , may be ignored in part or in total. Pushdown dependent join queries will be instances of Select with the relevant dependent sets available via Select.getDependentSets() . The dependent set is associated to Parameters by id via the Parameter.getDepenentValueId() identifier. The dependent set tuple iterators provide rows that are referenced by the column positions (available via Parameter.getValueIndex() ) on the dependent join Comparison criteria right expression. Care should be taken with the tuple values as they may guaranteed to be unique or ordered. Note There is no reference implementation of this functionality as all built-in translators rely on the engine to handle breaking up dependent joins into simpler queries. 3.2.5. Executing Commands 3.2.5.1. Execution Modes The Red Hat JBoss Data Virtualization query engine uses the ExecutionFactory class to obtain the Execution interface for the command it is executing. The query is sent to the translator as a set of objects. Refer to Section 3.2.2.1, "Language" for more information. Translators are allowed to support any subset of the available execution modes. Table 3.1. Types of Execution Modes Execution Interface Command interface(s) Description ResultSetExecution QueryExpression A query corresponding to a SQL SELECT or set query statement. UpdateExecution Insert, Update, Delete, BatchedUpdates An insert, update, or delete, corresponding to a SQL INSERT, UPDATE, or DELETE command ProcedureExecution Call A procedure execution that may return a result set and/or output values. All of the execution interfaces extend the base Execution interface that defines how executions are canceled and closed. ProcedureExecution also extends ResultSetExecution, since procedures may also return resultsets. 3.2.5.2. ExecutionContext The org.teiid.translator.ExecutionContext class provides information related to the current execution. An instance of ExecutionContext is available for each Execution . Various 'get' methods are provided; for example, ExecutionContext.getRequestIdentifier() and ExecutionContext.getSession() are provided for logging purposes. Specific usage is highlighted in this guide where applicable. 3.2.5.3. Generated Keys To see if the user query expects generated keys to be returned, consult the CommandContext.isReturnAutoGeneratedKeys() method. If you wish to return generated keys, you must first create a GeneratedKeys instance to hold the keys with the returnGeneratedKeys method passing the column names and types of the key columns. Only one GeneratedKeys may be associated with the CommandContext at any given time. 3.2.5.4. Source Hints The Red Hat JBoss Data Virtualization source meta-hint is used to provide hints directly to source executions via user or transformation queries. See the reference for more on source hints. If specified and applicable, the general and source specific hint will be supplied via the ExecutionContext methods getGeneralHint and getSourceHint . See the source for the OracleExecutionFactory for an example of how this source hint information can be utilized. 3.2.5.5. ResultSetExecution Typically most commands executed against translators are QueryExpression. While the command is being executed, the translator provides results via the ResultSetExecution.() method. This method returns null to indicate the end of results. Note: the expected batch size can be obtained using the ExecutionContext.getBatchSize() method and used as a hint in fetching results from the EIS. 3.2.5.6. Update Execution Each execution returns the update count(s) expected by the update command. If possible BatchedUpdates should be executed atomically. The ExecutionContext.isTransactional() method can be used to determine if the execution is already under a transaction. 3.2.5.7. Procedure Execution Procedure commands correspond to the execution of a stored procedure or some other functional construct. A procedure takes zero or more input values and can return a result set and zero or more output values. Examples of procedure execution would be a stored procedure in a relational database or a call to a web service. If a result set is expected when a procedure is executed, all rows from it will be retrieved via the ResultSetExecution interface first. Then, if any output values are expected, they will be retrieved using the getOutputParameterValues() method. 3.2.5.8. Asynchronous Executions In some scenarios, a translator will execute asynchronously and allow the executing thread to perform other work. To allow this, it is recommended that a DataNotAvailableException is thrown during a retrieval method, rather than explicitly waiting or sleeping for the results. Note The DataNotAvailableException should not be thrown by the execute method, as that can result in the execute method being called multiple times. The DataNotAvailableException may take a delay parameter or a Date in its constructor to indicate when to poll for results. Any non-negative delay value indicates the time in milliseconds until the polling should be performed. The DataNotAvailableException.NO_POLLING exception (or any DataNotAvailableException with a negative delay) can be thrown so that processing will resume (via ExecutionContext.dataAvailable() ). Since the execution (and the associated connection) is not closed until the work has completed, care must be taken if using asynchronous executions that hold a lot of state. A positive retry delay is not a guarantee of when the translator will be polled . If the DataNotAvailableException is consumed while the engine thinks more work can be performed or there are other shorter delays issued from other translators, then the plan may be queued again earlier than expected. You should throw a DataNotAvailableException again if your execution is not yet ready. Alternatively the DataNotAvailableException may be marked as strict, which does provide a guarantee that the Execution will not be called until the delay has expired or the given Date has been reached. Using the Date constructor makes the DataNotAvailableException automatically strict. Due to engine thread pool contention, platform time resolution, etc. a strict DataNotAvailableException is not a real-time guarantee of when the poll for results will occur, only that it will not occur before then. Note If your ExecutionFactory returns only asynch executions that perform minimal work, then consider having ExecutionFactory.isForkable return false so that the engine knows not to spawn a separate thread for accessing your Execution . 3.2.5.9. Reusable Executions A translator may return instances of ReusableExecutions for the expected Execution objects. There can be one ReusableExecution per query executing node in the processing plan. The lifecycle of a ReusableExecution is different that a normal Execution . After a normal creation/execute/close cycle the ReusableExecution.reset is called for the execution cycle. This may occur indefinitely depending on how many times a processing node executes its query. The behavior of the close method is no different from a regular Execution , it may not be called until the end of the statement if lobs are detected and any connection associated with the Execution will also be closed. When the user command is finished, the ReusableExecution.dispose() method will be called. In general ReusableExecutions are most useful for continuous query execution and will also make use of the ExecutionCotext.dataAvailable() method for Asynchronous Executions. See Red Hat JBoss Data Virtualization Development Guide: Client Development for more information about executing continuous statements. In continuous mode the user query will be continuously re-executed. A ReusableExecution allows the same Execution object to be associated with the processing plan for a given processing node for the lifetime of the user query. This can simplify asynch resource management, such as establishing queue listeners. Returning a null result from the () method ReusableExecution as with normal Executions indicates that the current pushdown command results have ended. Once the reset() method has been called, the set of results should be returned again terminated with a null result. See the kit examples for a reusable execution example. 3.2.5.10. Bulk Execution Non batched Insert, Update, Delete commands may have multi-valued Parameter objects if the capabilities shows support for BulkUpdate. Commands with multi-valued Parameters represent multiple executions of the same command with different values. As with BatchedUpdates, bulk operations should be executed atomically if possible. 3.2.5.11. Command Completion All normal command executions end with the calling of close() on the Execution object. Your implementation of this method should do the appropriate clean-up work for all state created in the Execution object. 3.2.5.12. Command Cancellation Commands submitted to Red Hat JBoss Data Virtualization may be aborted in several scenarios: Client cancellation via the JDBC API (or other client APIs) Administrative cancellation Clean-up during session termination Clean-up if a query fails during processing Unlike the other execution methods, which are handled in a single-threaded manner, calls to cancel happen asynchronously with respect to the execution thread. Your connector implementation may choose to do nothing in response to this cancellation message. In this instance, Red Hat JBoss Data Virtualization will call close() on the execution object after current processing has completed. Implementing the cancel() method allows for faster termination of queries being processed and may allow the underlying data source to terminate its operations faster as well. 3.2.6. Extending the Execution Factory Class 3.2.6.1. Extending the ExecutionFactory Class A custom translator must extend the org.teiid.translator.ExecutionFactory class to connect and query a data source. This extended class must provide a constructor with no arguments that can be constructed using Java reflection libraries. The following is an example constructor: package org.teiid.translator.custom; @Translator(name="custom", description="Connect to My EIS") public class CustomExecutionFactory extends ExecutionFactory<MyConnectionFactory, MyConnection> { public CustomExecutionFactory() { } } Specify the annotation @Translator on the extended "ExecutionFactory" class. This annotation defines the name and description of your translator, and is also used as an identifier during deployment. This is the name you would be using in the VDB and elsewhere in the configuration to refer to this translator. MyConnectionFactory specifies the type of ConnectionFactory interface that is expected from the associated resource adapter. This is required as part of the class definition when extending the ExecutionFactory class. MyConnection specifies the type of Connection interface that is expected from the associated resource adapter. This is required as part of class definition when extending the ExecutionFactory class. 3.2.6.2. Configuration Properties If the translator requires configurable properties then: define a variable for every property as an attribute in the extended ExecutionFactory class, define "get" and "set" methods for each attribute, and annotate each "get" method with @TranslatorProperty annotation and provide the metadata about the property. For example, if you need a property called foo , by providing the annotation on these properties, Red Hat JBoss Data Virtualization will automatically interrogate and provide a graphical way to configure your Translator while designing your VDB. private String foo = "blah"; @TranslatorProperty(display="Foo property", description="description about Foo") public String getFoo() { return foo; } public void setFoo(String value) { this.foo = value; } Only java primitive ( int , boolean ), primitive object wrapper ( java.lang.Integer ), or Enum types are supported as Translator properties. The default value will be derived from calling the getter, if available, on a newly constructed instance. All properties should have a default value. If there is no applicable default, then the property should be marked in the annotation as required. Initialization will fail if a required property value is not provided. The @TranslatorProperty defines the following metadata that you can define about your property. display - the display name of the property. description - a description about the property. required - specifies that the property is required. advanced - an advanced property (a default value must be provided). masked - tools need to mask the property, that is, do not show it in plain text. Used for passwords. Note A property can not be "advanced" and "required" at the same time. 3.2.6.3. Initializing the Translator Override and implement the start() method if your translator needs to do any initialization before it is used by the Red Hat JBoss Data Virtualization engine. This method must also call super.start() to perform any initialization required by the superclass. This method is called by Red Hat JBoss Data Virtualization once all the configuration properties are injected into the class. 3.2.6.4. Extended Translator Capabilities There are various methods, typically beginning with the method signature supports , that specify translator capabilities. These methods need to be overridden to describe the execution capabilities of the Translator. See Section 3.2.8.1, "Introduction to Translator Capabilities" for more information about these methods. 3.2.6.5. Execution (and sub-interfaces) Based on types of executions you are supporting, the following methods need to be overridden to provide implementations for their respective return interfaces. createResultSetExecution - Override if you are doing read based operation that is returning rows of results. For example, select. createUpdateExecution - Override if you are doing write based operations. For example, insert, update, delete. createProcedureExecution - Override if you are doing procedure based operations. For example, stored procedures. This works well for non-relational sources. You can choose to implement all the execution modes or only what you need. Refer to Section 3.2.5.1, "Execution Modes" for more information. 3.2.6.6. Metadata You can override and implement the method getMetadataProcessor() , in order to expose the metadata about the source for use in Dynamic VDBs. This defines the tables, column names, procedures, parameters, and so forth. for use in the query engine. This method is used by Designer tooling when the Teiid Connection importer is used. Here is a sample MetadataProcessor : If your MetadataProcessor needs external properties that are needed during the import process, you can define them on the MetadataProcessor . For example, to define a import property called "Column Name Pattern", which can be used to filter which columns are defined on the table, you can add it like this: Note the category type. The configuration property defined in the section is different from this one. Configuration properties define the runtime behavior of translator, where as "IMPORT" properties define the metadata import behavior, and aid in controlling what metadata is exposed by your translator. These properties can be automatically injected through "import" properties set through Designer when using the "Teiid Connection" importer or the properties can be defined under the model construct in the vdb.xml file, like There may be times when implementing a custom translator, the built in metadata about your schema is not enough to process the incoming query due to variance of semantics with your source query. To aid this issue, Teiid provides a mechanism called "Extension Metadata", which is a mechanism to define custom properties and then add those properties on metadata object (table, procedure, function, column, index and so forth). For example, in a custom translator a table might represent a file on disk. In this case, you could define an extension metadata property like this: The @ExtensionMetadataProperty defines the following metadata that you can define about your property: applicable : Metadata object this is applicable on. This is array of metadata classes like Table.class and Column.class. datatype : The java class indicating the data type display : Display name of the property description : Description about the property required : Indicates if the property is a required property When you define an extension metadata property like above, during the runtime you can obtain the value of that property. If you get the query object which contains 'SELECT * FROM MyTable' , MyTable will be represented by an object called NamedTable . Now you have accessed the file name you set during the construction of the Table schema object, and you can use this value however you seem feasible to execute your query. With the combination of built in metadata properties and extension metadata properties you can design and execute queries for a variety of sources. 3.2.6.7. Logging Red Hat JBoss Data Virtualization provides the org.teiid.logging.LogManager class for logging purposes, based on the Apache Log4j logging services. Logging messages will be sent automatically to the main Red Hat JBoss Data Virtualization logs. You can customize logging by editing the corresponding subsystem in the server configuration file or via the Management Console. 3.2.6.8. Exceptions When throwing exceptions in translator code, use the org.teiid.translator.TranslatorException class. 3.2.6.9. Default Name You can define a default instance of your Translator by defining the annotation @Translator on the ExecutionFactory . After deployment, a default instance of this Translator can be used by any VDB by referencing it by this name in its vdb.xml configuration file. A VDB can also override the default properties and define another instance of this translator too. The name you give here is the short name used everywhere else in the Red Hat JBoss Data Virtualization configuration to refer to this translator. Note The translator created here is only available in the scope of the VDB - it is not available to the whole Red Hat JBoss Data Virtualization instance. 3.2.6.10. Obtaining Connections The extended ExecutionFactory must implement the getConnection() method to allow the Connector Manager to obtain a connection. 3.2.6.11. Releasing Connections Connections are only used for the lifetime of the request. When the request completes, the closeConnection() method is called on the ExecutionFactory . You must override this method to close the connection properly. If the resource adapter is JEE JCA Connector based, connection pooling is automatically provided. Note Red Hat recommends the use of connection pooling when a connection is stateful or when connections are expensive to create. 3.2.7. Large Objects 3.2.7.1. Data Types Red Hat JBoss Data Virtualization supports three large object runtime data types: BLOB, CLOB, and XML. A BLOB is a "binary large object", a CLOB is a "character large object", and XML is an "xml document". Columns modeled as a BLOB, CLOB, or XML are treated similarly by the translator framework to support memory-safe streaming. 3.2.7.2. Why Use Large Object Support? Red Hat JBoss Data Virtualization allows a Translator to return a large object through the translator API by returning a reference to the actual large object. Access to that LOB will be streamed as appropriate rather than retrieved all at once. This is useful for several reasons: Reduces memory usage when returning the result set to the user. Improves performance by passing less data in the result set. Allows access to large objects when needed rather than assuming that users will always use the large object data. Allows the passing of arbitrarily large data values. However, these benefits can only truly be gained if the Translator itself does not materialize an entire large object all at once. For example, the Java JDBC API supports a streaming interface for BLOB and CLOB data. 3.2.7.3. Handling Large Objects The Translator API automatically handles large objects (BLOB/CLOB/SQLXML) through the creation of special purpose wrapper objects when it retrieves results. Once the wrapped object is returned, the streaming of LOB is automatically supported. These LOB objects can then, for example, appear in client results, in user defined functions, or be sent to other translators. An Execution is usually closed and the underlying connection is either closed/released as soon as all rows for that execution have been retrieved. However, LOB objects may need to be read after their initial retrieval of results. When LOBs are detected the default closing behavior is prevented by setting a flag using the ExecutionContext.keepAlive() method. When the "keepAlive" flag is set, then the execution object is only closed when user's Statement is closed. executionContext.keepExecutionAlive(true); 3.2.7.4. Inserting or Updating Large Objects LOBs will be passed to the Translator in the language objects as Literal containing a java.sql.Blob, java.sql.Clob, or java.sql.SQLXML. You can use these interfaces to retrieve the data in the large object and use it for insert or update. 3.2.8. Translator Capabilities 3.2.8.1. Introduction to Translator Capabilities The ExecutionFactory class defines all the methods that describe the capabilities of a Translator. These are used by the Connector Manager to determine what kinds of commands the translator is capable of executing. A base ExecutionFactory class implements all the basic capabilities methods, which says your translator does not support any capabilities. Your extended ExecutionFactory class must override the necessary methods to specify which capabilities your translator supports. You should consult the debug log of query planning (set showplan debug) to see if desired pushdown requires additional capabilities. Note Your capabilities will remain unchanged for the lifetime of the translator, since the engine will cache them for reuse by all instances of that translator. Capabilities based on connection/user are not supported. 3.2.8.2. Translator Development During translator development, you can define three different types of property sets that can help customize the behavior of the translator. On the "ExecutionFactory" class a translator developer can define any number of "getter/setter" methods with the @TranslatorProperty annotation. These properties (also referred to a execution properties) can be used for extending the capabilities of the translator. It is important to define default values for all these properties, as these properties are being defined to change the default behavior of the translator. If needed, the values for these properties are supplied in "vdb.xml" file during the deploy time when the translator is used to represent vdb's model. Here is an example: @TranslatorProperty(display="Copy LOBs",description="If true, returned LOBs will be copied, rather than streamed from the source",advanced=true) public boolean isCopyLobs() { return copyLobs; } public void setCopyLobs(boolean copyLobs) { this.copyLobs = copyLobs; } At runtime, you can define these properties in the vdb.xml file like this: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <vdb name="vdb" version="1"> <model name="PM1"> <source name="connector" translator-name="my-translator-override" /> </model> <translator name="my-translator-override" type="my-translator"> <property name="CopyLobs" value="true" /> </translator> </vdb> If a translator is defining schema information based on the physical source (i.e. implementing getMetadata method on ExecutionFactory) it is connected to, then import properties provide a way to customize the behavior of the import process. For example, in the JDBC translator users can exclude certain tables that match a regular expression etc. To define an import property, the @TranslatorPropery annotation is used on any getter/setter method on the "ExecutionFactory" class or any class that implements the "MetadataProcessor" interface, with category property defined as "PropertyType.IMPORT": @Translator(name = "my-translator", description = "My Translator") public class MyExecutionFactory extends ExecutionFactory<ConnectionFactory, MyConnection> { ... public MetadataProcessor<C> getMetadataProcessor() { return MyMetadataProcessor(); } } public MyMetadataProcessor implements MetadataProcessor<MyConnection> { public void process(MetadataFactory metadataFactory, MyConnection connection) throws TranslatorException{ // schema generation code here } @TranslatorProperty(display="Header Row Number", category=PropertyType.IMPORT, description="Row number that contains the header information") public int getHeaderRowNumber() { return headerRowNumber; } public void setHeaderRowNumber(int headerRowNumber) { this.headerRowNumber = headerRowNumber; } } This is how you use import properties with a vdb.xml file <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <vdb name="vdb" version="1"> <model name="PM1"> <property name="importer.HeaderRowNumber" value="12"/> <source name="connector" translator-name="my-translator" /> </model> </vdb> Note When properties are defined using the annotation mechanism and also when you use the "Teiid Connection" importer in the Designer, these properties will automatically show up in the wizard's relevant input field. During the execution of the command, the translator is responsible for converting Data Virtualization-supplied SQL commands into data-source specific queries. There are many cases when the built-in metadata is not sufficient and additional metadata about the source is useful to form a request for the underlying physical source system. Extension Metadata Properties are one such mechanism that can be used to fill in the gaps in the metadata. You can define specific properties for a given translator. The translator communicates with the query engine about its source through its metadata. Metadata in this context consists of definitions of Tables, Columns, Procedures, Keys, and so forth. This metadata can be decorated with additional custom metadata and fed into the query engine. The query engine keeps this extended metadata intact along with its schema objects, and when a user query is submitted to the the translator for execution, this extended metadata can be retrieved for making decisions in the translator code. Extended properties are defined using annotation class called @ExtensionMetadataProperty on the fields in your "MetadataProcessor" or "ExecutionFactory" classes. For example, say translator requires a "encoding" property on Table, to do the correct un-marshaling of data, this property can be defined like this: public class MyMetadataProcessor implements MetadataProcessor<MyConnection> { public static final String URI = "{http://www.teiid.org/translator/mytranslator/2014}"; @ExtensionMetadataProperty(applicable=Table.class, datatype=String.class, display="Encoding", description="Encoding", required=true) public static final String ENCODING = URI+"encode"; public void process(MetadataFactory mf, FileConnection conn) throws TranslatorException { .. Table t = mf.addTable(tableName); t.setProperty(ENCODING, "UTF-16"); // add columns etc. .. } } Now during the execution, on the COMMAND object supplied to the "Execution" class, user can do this: Select select = (Select)command; NamedTable tableReferece = select.getFrom().get(0); Table t = tableReference.getMetadataObject(); String encoding = t.getProperty(MyMetadataProcessor.ENCODING, false); // use the encoding value as needed to marshal or unmarshal data Note When extended properties are defined using the annotation mechanism, when using "Teiid Connection" importer in the Designer, you do not need to define the "Metadata Extension Defn" in designer and register to use with your model, the required definitions are automatically downloaded and configured to use. 3.2.8.3. ExecutionFactory Class Capabilities Table 3.2. Available Capabilities Capability Requires Description SelectDistinct The translator supports SELECT DISTINCT in queries. SelectExpression The translator supports SELECT of more than column references. AliasedTable The translator supports Tables in the FROM clause that have an alias. InnerJoins The translator supports inner and cross joins. SelfJoins AliasedGroups and at least on of the join type supports. The translator supports a self join between two aliased versions of the same Table. OuterJoins The translator supports LEFT and RIGHT OUTER JOIN. FullOuterJoins The translator supports FULL OUTER JOIN. DependentJoins Base join and criteria support The translator supports key set dependent join pushdown (see Section 3.2.4, "Dependent Join Pushdown" ). When set, the MaxDependentInPredicates and MaxInCriteriaSize values are not used by the engine, rather all independent values are made available to the pushdown command. SubqueryInOn Join and base subquery support, such as ExistsCriteria The translator supports subqueries in the ON clause. It defaults to true. InlineViews AliasedTable The translator supports a named subquery in the FROM clause. BetweenCriteria This is not currently used - between criteria are rewritten as compound comparisons. CompareCriteriaEquals The translator supports comparison criteria with the operator "=". CompareCriteriaOrdered The translator supports comparison criteria with the operator ">" or "<". LikeCriteria The translator supports LIKE criteria. LikeCriteriaEscapeCharacter LikeCriteria The translator supports LIKE criteria with an ESCAPE character clause. SimilarTo The translator supports SIMILAR TO criteria. LikeRegexCriteria The translator supports LIKE_REGEX criteria. InCriteria MaxInCriteria The translator supports IN predicate criteria. InCriteriaSubquery The translator supports IN predicate criteria where values are supplied by a subquery. IsNullCriteria The translator supports IS NULL predicate criteria. OrCriteria The translator supports the OR logical criteria. NotCriteria The translator supports the NOT logical criteria. IMPORTANT: This capability also applies to negation of predicates, such as specifying IS NOT NULL, "<=" (not ">"), ">=" (not "<"), etc. ExistsCriteria The translator supports EXISTS predicate criteria. QuantifiedCompareCriteriaAll The translator supports a quantified comparison criteria using the ALL quantifier. QuantifiedCompareCriteriaSome The translator supports a quantified comparison criteria using the SOME or ANY quantifier. OnlyLiteralComparison If only Literal comparisons (equality, ordered, like, and so on) are supported for non-join conditions. Convert(int fromType, int toType) This is used for fine-grained control of the convert/cast pushdown. The ExecutionFactory.getSupportedFunctions() must contain SourceSystemFunctions.CONVERT . This method can then return false to indicate a lack of specific support. (See TypeFacility.RUNTIME_CODES for the possible type codes.) The engine does not care about unnecessary conversions where fromType == toType . By default lob conversion is disabled. OrderBy The translator supports the ORDER BY clause in queries. OrderByUnrelated OrderBy The translator supports ORDER BY items that are not directly specified in the select clause. OrderByNullOrdering OrderBy The translator supports ORDER BY items with NULLS FIRST/LAST. GroupBy The translator supports an explicit GROUP BY clause. Having GroupBy The translator supports the HAVING clause. AggregatesAvg The translator supports the AVG aggregate function. AggregatesCount The translator supports the COUNT aggregate function. AggregatesCountStar The translator supports the COUNT(*) aggregate function. AggregatesDistinct At least one of the aggregate functions. The translator supports the keyword DISTINCT inside an aggregate function. This keyword indicates that duplicate values within a group of rows will be ignored. AggregatesMax The translator supports the MAX aggregate function. AggregatesMin The translator supports the MIN aggregate function. AggregatesSum The translator supports the SUM aggregate function. AggregatesEnhancedNumeric The translator supports the VAR_SAMP, VAR_POP, STDDEV_SAMP, STDDEV_POP aggregate functions. ScalarSubqueries The translator supports the use of a subquery in a scalar context (wherever an expression is valid). CorrelatedSubqueries At least one of the subquery pushdown capabilities. The translator supports a correlated subquery that refers to an element in the outer query. CaseExpressions This is not currently used - simple case is rewritten as searched case. SearchedCaseExpressions The translator supports "searched" CASE expressions anywhere that expressions are accepted. Unions The translator supports UNION and UNION ALL. Intersect The translator supports INTERSECT. Except The translator supports EXCEPT. SetQueryOrderBy Unions, Intersect, or Except The translator supports set queries with an ORDER BY. RowLimit The translator supports the limit portion of the limit clause. RowOffset The translator supports the offset portion of the limit clause. FunctionsInGroupBy GroupBy The translator supports non-column reference grouping expressions. InsertWithQueryExpression Translator supports INSERT statements with values specified by a QueryExpression. BatchedUpdates The translator supports a batch of INSERT, UPDATE and DELETE commands to be executed together. BulkUpdate Translator supports updates with multiple value sets CommonTableExpressions The translator supports the WITH clause. ElementaryOlapOperations The translator supports window functions and analytic functions RANK, DENSE_RANK, and ROW_NUMBER. WindowOrderByWithAggregates ElementaryOlapOperations The translator supports windowed aggregates with a window order by clause. WindowDistinctAggregates ElementaryOlapOperations, AggregatesDistinct The translator supports windowed distinct aggregates. AdvancedOlapOperations ElementaryOlapOperations The translator supports aggregate conditions. OnlyFormatLiterals This provides function support for a parse/format function and an implementation of the supportsFormatLiteral method. The translator supports only literal format patterns that are validated by the supportsFormatLiteral method. FormatLiteral(String literal, Format type) OnlyFormatLiterals The translator supports the given literal format string. ArrayType The translator supports the push down of array values. OnlyCorrelatedSubqueries CorrelatedSubqueries The translator ONLY supports correlated subqueries. Uncorrelated scalar and exists subqueries will be pre-evaluated prior to push-down. SelectWithoutFrom SelectExpressions The translator supports selecting values without a FROM clause, such as SELECT 1. Note Note that any pushdown subquery must itself be compliant with the translator's capabilities. 3.2.8.4. Command Form The method ExecutionFactory.useAnsiJoin() should return true if the Translator prefers the use of ANSI style join structure for join trees that contain only INNER and CROSS joins. The method ExecutionFactory.requiresCriteria() should return true if the Translator requires criteria for any Query, Update, or Delete. This is a replacement for the model support property "Where All". 3.2.8.5. Scalar Functions The method ExecutionFactory.getSupportedFunctions() can be used to specify which scalar and aggregate functions the Translator supports. The set of possible functions is based on the set of functions supported by Red Hat JBoss Data Virtualization. This set can be found in the Red Hat JBoss Data Virtualization Reference Guide. If the Translator states that it supports a function, it must support all type combinations and overloaded forms of that function. There are also some standard operators that can be specified in the supported function list: +, -, *, and /. The constants interface SourceSystemFunctions contains the string names of all possible built-in pushdown functions. Note that not all system functions appear in this list. This is because some system functions will always be evaluated in Red Hat JBoss Data Virtualization, are simple aliases to other functions, or are rewritten to a more standard expression. A translator may also indicate support for scalar functions that are intended for pushdown evaluation by that translator, but are not registered as user defined functions via a model/schema. These pushdown functions are reported to the engine via the ExecutionFactory.getPushDownFunctions() list as FunctionMethod metadata objects. The FuncitonMethod representation allow the translator to control all of the metadata related to the function, including type signature, determinism, varargs, etc. The simplest way to add a pushdown function is with a call to ExecutionFactory.addPushDownFunction : FunctionMethod addPushDownFunction(String qualifier, String name, String returnType, String...paramTypes) This resulting function will be known as sys.qualifier.name, but can be called with name only as long as the function name is unique. The returned FunctionMethod object may be further manipulated depending upon the needs of the source. An example of adding a custom concat vararg function in an ExecutionFactory subclass: public void start() throws TranslatorException { super.start(); FunctionMethod func = addPushDownFunciton("oracle", "concat", "string", "string", "string"); func.setVarArgs(true); ... } 3.2.8.6. Physical Limits The method ExecutionFactory.getMaxInCriteriaSize() can be used to specify the maximum number of values that can be passed in an IN criteria. This is an important constraint as an IN criteria is frequently used to pass criteria between one source and another using a dependent join. The method ExecutionFactory.getMaxDependentInPredicates() is used to specify the maximum number of IN predicates (of at most MaxInCriteriaSize) that can be passed as part of a dependent join. For example if there are 10000 values to pass as part of the dependent join and a MaxInCriteriaSize of 1000 and a MaxDependentInPredicates setting of 5, then the dependent join logic will form two source queries each with 5 IN predicates of 1000 values each combined by OR. The method ExecutionFactory.getMaxFromGroups() can be used to specify the maximum number of FROM Clause groups that can used in a join. -1 indicates there is no limit. 3.2.8.7. Update Execution Modes The method ExecutionFactory.supportsBatchedUpdates() can be used to indicate that the Translator supports executing the BatchedUpdates command. The method ExecutionFactory.supportsBulkUpdate() can be used to indicate that the Translator accepts update commands containg multi valued Literals. Note Note that if the translator does not support either of these update modes, the query engine will compensate by issuing the updates individually. 3.2.8.8. Null Ordering The method ExecutionFactory.getDefaultNullOrder() specifies the default null order. It can be one of UNKNOWN, LOW, HIGH, FIRST, LAST. This is only used if ORDER BY is supported, but null ordering is not. The method ExecutionFactory.getCollation() specifies the default collation. If you set it to a value that does not match the collation locale defined by org.teiid.collationLocale, then some ordering may not be pushed down.

Commands:
//getting the Table metadata from an Table is straight-forward Table table = runtimeMetadata.getTable("table-name"); String contextName = table.getNameInSource(); //The props will contain extension properties Map<String, String> props = table.getProperties();
// Get object tree LanguageObject objectTree = // Create your visitor initialize as necessary MyVisitor visitor = new MyVisitor(); // Call the visitor using pre-order visitation DelegatingHierarchyVisitor.preOrderVisit(visitor, objectTree); // Retrieve state collected while visiting int count = visitor.getCount();
package org.teiid.translator.custom; @Translator(name="custom", description="Connect to My EIS") public class CustomExecutionFactory extends ExecutionFactory<MyConnectionFactory, MyConnection> { public CustomExecutionFactory() { } }
private String foo = "blah"; @TranslatorProperty(display="Foo property", description="description about Foo") public String getFoo() { return foo; } public void setFoo(String value) { this.foo = value; }
public class MyMetadataProcessor implements MetadataProcessor<Connection> { public void process(MetadataFactory mf, Connection conn) { Object somedata = connection.getSomeMetadata(); Table table = mf.addTable(tableName); Column col1 = mf.addColumn("col1", TypeFacility.RUNTIME_NAMES.STRING, table); Column col2 = mf.addColumn("col2", TypeFacility.RUNTIME_NAMES.STRING, table); //add a pushdown function that can also be evaluated in the engine Method method = FunctionMethod f = mf.addFunction("func", method); //add a pushdown aggregate function that can also be evaluated in the engine Method aggMethod = FunctionMethod af = mf.addFunction("agg", aggMethod); af.setAggregateAttributes(new AggregateAttributes()); } }
@TranslatorProperty(display="Column Name Pattern", category=PropertyType.IMPORT, description="Pattern to derive column names") public String getColumnNamePattern() { return columnNamePattern; } public void setColumnNamePattern(String columnNamePattern) { this.columnNamePattern = columnNamePattern; }
<vdb name="myvdb" version="1"> <model name="legacydata" type="PHYSICAL"> <property name="importer.ColumnNamePattern" value="col*"/> . <source name = .../> </model> </vdb>
public class MyMetadataProcessor implements MetadataProcessor<Connection> { public static final String NAMESPACE = "{http://my.company.corp}"; @ExtensionMetadataProperty(applicable={Table.class}, datatype=String.class, display="File name", description="File Name", required=true) public static final String FILE_PROP = NAMESPACE+"FILE"; public void process(MetadataFactory mf, Connection conn) { Object somedata = connection.getSomeMetadata(); Table table = mf.addTable(tableName); table.setProperty(FILE_PROP, somedata.getFileName()); Column col1 = mf.addColumn("col1", TypeFacility.RUNTIME_NAMES.STRING, table); Column col2 = mf.addColumn("col2", TypeFacility.RUNTIME_NAMES.STRING, table); } }
for (TableReference tr:query.getFrom()) { NamedTable t = (NamedTable) tr; Table table = t.getMetadataObject(); String file = table.getProperty(FILE_PROP); .. }
executionContext.keepExecutionAlive(true);
@TranslatorProperty(display="Copy LOBs",description="If true, returned LOBs will be copied, rather than streamed from the source",advanced=true) public boolean isCopyLobs() { return copyLobs; } public void setCopyLobs(boolean copyLobs) { this.copyLobs = copyLobs; }
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <vdb name="vdb" version="1"> <model name="PM1"> <source name="connector" translator-name="my-translator-override" /> </model> <translator name="my-translator-override" type="my-translator"> <property name="CopyLobs" value="true" /> </translator> </vdb>
@Translator(name = "my-translator", description = "My Translator") public class MyExecutionFactory extends ExecutionFactory<ConnectionFactory, MyConnection> { public MetadataProcessor<C> getMetadataProcessor() { return MyMetadataProcessor(); } } public MyMetadataProcessor implements MetadataProcessor<MyConnection> { public void process(MetadataFactory metadataFactory, MyConnection connection) throws TranslatorException{ // schema generation code here } @TranslatorProperty(display="Header Row Number", category=PropertyType.IMPORT, description="Row number that contains the header information") public int getHeaderRowNumber() { return headerRowNumber; } public void setHeaderRowNumber(int headerRowNumber) { this.headerRowNumber = headerRowNumber; } }
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <vdb name="vdb" version="1"> <model name="PM1"> <property name="importer.HeaderRowNumber" value="12"/> <source name="connector" translator-name="my-translator" /> </model> </vdb>
public class MyMetadataProcessor implements MetadataProcessor<MyConnection> { public static final String URI = "{http://www.teiid.org/translator/mytranslator/2014}"; @ExtensionMetadataProperty(applicable=Table.class, datatype=String.class, display="Encoding", description="Encoding", required=true) public static final String ENCODING = URI+"encode"; public void process(MetadataFactory mf, FileConnection conn) throws TranslatorException { .. Table t = mf.addTable(tableName); t.setProperty(ENCODING, "UTF-16"); // add columns etc. .. } }
Select select = (Select)command; NamedTable tableReferece = select.getFrom().get(0); Table t = tableReference.getMetadataObject(); String encoding = t.getProperty(MyMetadataProcessor.ENCODING, false); // use the encoding value as needed to marshal or unmarshal data
FunctionMethod addPushDownFunction(String qualifier, String name, String returnType, String...paramTypes)
public void start() throws TranslatorException { super.start(); FunctionMethod func = addPushDownFunciton("oracle", "concat", "string", "string", "string"); func.setVarArgs(true); }