OpenLLM-France/wikimedia huggingface dataset:
# XQuery/XQuery and XML Schema
## Motivation

In learning any modelling language, it is helpful to see sample instances of any formal model. This is equally true when developing an XML Schema. XML Development tools like Oxygen and XML-Spy have tools to generate a random instance of a supplied XML schema, but it is useful to have a web service to do this. The service can also be used for test data generation.
The following is a test XML Schema file:

## XQuery to convert Simple XML Schema into an Instance Document

We can use a recursive function to convert this XML Schema into an instance document. Here is an implementation of this function which covers some of the key XML Schema constructs:
This function uses the element construct to create a new XML instance tree based on the information in the XML Schema file and the typeswitch construct to select the appropriate processing of a given element type.

## Type Instance Generation

Some helper functions are required to generate random instances of (some of ) the standard types.

## Script

Here is a basic script which applies this function to the example schema. By default, the root of the generated instance is the first element in the schema or a named element if a root parameter is supplied.
Execute

## To do

* full set of xml types
* mixed
* restriction (only enumeration so far)
* Group
* AttributeGroup
* problem with missing attributes in a complexType
* hinting for distributions
* namespaces
* randomisation configuration
# Motivation

Adobe have introduced an XML format for image metadata called XMP. You want to display a photograph and some of the metadata.

# Background

Matt Turner has an example of using MarkLogic to extract XMP data from a JPEG image .

# eXist/ XQuery implementation

Here is the code using eXist XQuery extensions. Matt's " photograph of a boat has been stored in the XML Database as a binary file.

# Show the full XMP XML

The binary image is retrieved from the database as a BASE64 encoded string, converted to a UTF-8 string, the metadata text extracted and turned back to XML using util:parse
XMP XML

# Some basic Dublin Core elements

Here we extract a few of the Dublin Core elements and create an HTML fragment containing the image and the meta-data.
Basic Dublin Core elements
# XQuery/XPath examples
## Motivation

You would like to select specific structures within an XML document. You would like to use a language that is consistent across all W3C XML Standards. This language is XPath.

## Sample Input File

Put this file in /db/apps/training/data/books.xml
XPath provides a number of functions and axes to move around an XML structure.

## Sample Test XQuery Script

Here is a sample XQuery "driver" for these tests. To use it just replace the function after the return on the last line.

## Screen Image of XQuery Results in oXygen

There are several ways to test your XPath expressions. One of the most useful is to put your XML test data in document within eXist and then use a tool such as oXygen to execute the test on the server but then display the results in the oXygen results window. This can be done within oXygen by setting up eXist (not the default internal Saxon) as your "transformation scenario".
The following is a screen image of how these results look when viewed with the oXygen IDE:

## Sample XPath Expressions

Return the entire data file
Get just books and all the data for each book.
Get all the book titles
Get the collection description
Get the descriptions for all the books

### Counting and Math

Count the number of books using a absolute path
Count the number of books using //. With eXist executes much faster on larger collections.
Get a sequence of all the titles in the book collection.
Calculate the total and average price of all the books in the collection.
The following scripts show some of these functions and axes in use.

### Adding Predicates

A predicate is a qualifier that is added to the end of an XPath expression. It is usually used to filter out nodes from result set. Predicates are similar to WHERE constructs in SQL.
Get all the wikibooks
Get just the titles of the wikibooks
Get all the books that contain the word "XQuery" somewhere in the title

### Complex Predicates

* node
* text
* string(..)
* data(..)
* child::
* parent::
* following-sibling::
* preceding-sibling::
* descendant::
* descendant-or-self
Navigating around a tree with distinct tags
Navigating around a tree with a single tag
# XQuery/XQuery Batch Jobs
## Motivation

You want to run an XQuery job at regular intervals.

## Method

We will use the eXist job scheduler. The eXist job scheduler is built around the Quartz system and eXist provides an XQuery API to this system to add and remove jobs.

### Method 1: Modify the conf.xml file

If you have a job that needs to run on a regular basis, you can just add a single line to your $EXIST_HOME/conf.xml file. For example, if you have a simple XQuery script that writes a dateTime stamp to the log file, you could add the following line:

#### Sample Addition to conf.xml

This line says that when seconds=0 for every minute of every hour of every day-of-month for every month for each day of week, run this job.

#### Sample Weekly Lucene Optimize

Add the following line to your $EXIST_HOME/conf.xml file.

#### Sample XQuery to Optimize Lucene Indexes

Contents of /db/system/jobs/optimize-lucene-indexes.xq

### Method 2: Use the XQuery API

In this method we will use the XQuery API to add, view and remove jobs from the job scheduler.
To enable the XQuery scheduler you may have to set a line in the $EXIST_HOME/extensions
And then type "build" to recompile the code.
And also make sure that the line in the $EXIST_HOME/conf.xml is un-commented.
Here are the two functions to add and delete jobs.
Note: You must make sure that the XQuery job scheduler module is enabled in your system. You can verify this by the following XQuery:
The format of cron string is documented :

### Listing Scheduled Jobs

You can get a list of all scheduled jobs by using the scheduler:get-scheduled-jobs XQuery function. This returns a document that has the following format:

### Adding and Removing Jobs with XQuery

The following is a sample of the system calls to add and remove jobs:

## Avoiding Concurrent Jobs

Sometimes you want to run a job frequently that polls a remote site, for example once every five minutes. If it finds a file it might want to transfer files. But sometimes the time to transfer the files is longer than the polling frequency. This will restart the job again.
To get around this you have two options. One is to be able to configure eXist to not run concurrent jobs. This is not available in 2.1. In that case you may need to set a flag that will test to see if the prior job has finished. You can use the cache module to set one flag for each job.
The following shows how you can use the put/get and remove functions to manage global state across queries:
# XQuery/XQuery and XSLT
## Motivation

You want to create a RESTful web service that executes an XSLT transform on an XML document.

## Method

XQuery is superior to XSLT in many ways. XQuery is designed to be brief and concise programming language that interleaves XML and functional language statements. Therefore, XQuery programs are usually much smaller than XSLT. XQuery processors are also designed to use indexes so that XQueries over large data sets can run quickly. But unfortunately there are still some times when you must use XSLT. One example of this is in-browser transforms. The eXist database comes with an XQuery function that allows you to transform an XML file using XSLT.

## Creating an XSLT service

eXist includes a function to call an XSLT transform is the following:
where:
The result is zero or one nodes. The namespace of the transform module is http://exist-db.org/xquery/transform.
The transform:transform function can be used to provide a service which accepts the url of an XML file, the url of an XSLT script and any other parameters which are passed to the stylesheet.
Currently output is text/html.

### Checking XSLT Version

The following XSLT is useful for checking what version of XSLT you are running.

### Form-based search

In this example, an XML file on one host is transformed by a XSLT script on another. The XSLT script defines a form to allow the use to select a subset of the entries in the XML file, followed by the search results, if any.
* Stylesheet
* Data
* Search Whisky data
A sequence diagram describes the interaction involved:
Sequence Diagram

### Page Scraping Example

see Page scraping and Yahoo Weather

### Using XSLT Imports

XSLT allows you to call a stylesheet that imports a common library of other XSLT templates. But not all of the XSLT import path statements will work within eXist. In the following example we will use a XSLT style sheet that imports another style sheet. The following assumes you have a collection called /db/test/xslt and all the files are placed in that collection.

#### Top-Level Style.xsl

In the second line, the following imports do work as expected:
But you will note that using the following does not work:

#### Imported common.xsl
## XForms Example

You can also create a simple XForms example that serves as a front end to this script. See the XRX wikibook for an example of this XForms front end.

## Caching Management

By default, once a document has been transformed it resides in the cache. This is very good for performance reasons if a file needs to be retransformed but sometimes if the source file changes the transform needs to be rerun.
You can disable caching by changing the configuration file. In the file conf.xml change the @caching value from yes to no.:
http://demo.exist-db.org/exist/xquery.xml#N10375
# XQuery/XSL-FO Images
## Motivation

You want to enrich your documents with print quality images and charts etc.

## Method

We will use the fo:external-graphic primitive.
For example, to add an external image add a block to the XSL-FO:
execute

## Vector Images

SVG is a standard way to describe graphical artwork as vectors. Recent eXist installations (>1.4) with the Apache FOP processor enabled can embed SVG data in the resulting PDF as vector art: just reference them via http redirection as they are not in the file system. See Generating PDF from XSL-FO files on how to activate the XSLFO feature.

## PDF image extension

For the daring: there is an extension to apache fop, that provides for pdf-images: a method of placing pages of PDF files in the FOP output. Its the work of Jeremias Märki and available at his webspace.
I had to restart exist to activate pdf-images support in fop. The fo syntax is the same as with SVG, a page-number can be specified after a hash sign in the URL.
# XQuery/XQuery IDE

In progress
The eXist Database stores binary files as well as XML files. Binary files include the XQuery scripts themselves. This allows XQuery scripts to manipulate the XQuery scripts themselves - viewing, searching, analyzing, modifying and creating scripts, all the operations required of a development environment.

## Viewing the XQuery script

The eXist util module includes functions to read binary documents and convert to text. Here they are used to view a script. In this public interface, a configuration file is used to control access to those scripts.
The script checks that a requested script name is present in the list of public scripts before retrieving the script.
# XQuery/XQuery SQL Module
## Motivation

You would like to perform SQL queries from within your XQuery code.

## Method

eXistdb provides a module for executing SQL queries.

## Configuration Steps

* Enable the module
* Configure your connection string
* Execute a test query

### Enable the SQL Module

Your fist step is to enable the SQL Module. To do this you must uncomment the following lines from the conf.xml file in your EXIST_HOME directory:
In eXist 1.5 there is also an additional Oracle module that is undocumented.

### Install the Driver library

The SQL Module uses JDBC for its database connectivity and as such for each database type that you wish to connect to, a JDBC Driver is required. JDBC Drivers should be placed in EXIST_HOME/lib/user.
After this is done you must restart your server. You should now see the additional SQL Module documentation in your function list and the Driver class(es) used in the connection function calls should be found.

### Execute the Query

In order to execute the query there are two steps you must take:
* get a connection to the database
* execute the query
There are five different functions to get a connection to the database but only one function to execute the query.
The connection string allows you to connect to the correct server with the appropriate username and password.
In its most basic form, the format of the get-connection function is the following:
This format assumes you can put the login and password to the database directly in the JDBC connection URL.
If you can not do this, the format of the connection string with a username and password is:
Note that some systems also put the username and password in the JDBC connection string.
For example in MySQL the string might be:
In Oracle the string might be

### References

eXistdb Documentation
# This Quantum World/Implications and applications/A quantum bouncing ball

< PREVIOUS

## A quantum bouncing ball

As a specific example, consider the following potential:
    $V(z)=mgz\quad {\hbox{if}}\quad z>0\quad {\hbox{and}}\quad V(z)=\infty \quad {\hbox{if}}\quad z<0.$
$g$ is the gravitational acceleration at the floor. For $z<0,$ the Schrödinger equation as given in the previous section tells us that $d^{2}\psi (z)/dz^{2}=\infty$ unless $\psi (z)=0.$ The only sensible solution for negative $z$ is therefore $\psi (z)=0.$ The requirement that $V(z)=\infty$ for $z<0$ ensures that our perfectly elastic, frictionless quantum bouncer won't be found below the floor.
Since a picture is worth more than a thousand words, we won't solve the time-independent Schrödinger equation for this particular potential but merely plot its first eight solutions:
Where would a classical bouncing ball subject to the same potential reverse its direction of motion? Observe the correlation between position and momentum (wavenumber).
All of these states are stationary; the probability of finding the quantum bouncer in any particular interval of the $z$ axis is independent of time. So how do we get it to move?
Recall that any linear combination of solutions of the Schrödinger equation is another solution. Consider this linear combination of two stationary states:
    $\psi (t,x)=A\,\psi _{1}(x)\,e^{-i\omega _{1}t}+B\,\psi _{2}(x)\,e^{-i\omega _{2}t}.$
Assuming that the coefficients $A,B$ and the wave functions $\psi _{1}(x),\psi _{2}(x)$ are real, we calculate the mean position of a particle associated with $\psi (t,x)$:
    $\int \!dx\,\psi ^{*}x\psi =\int \!dx\,(A\psi _{1}e^{i\omega _{1}t}+B\psi _{2}e^{i\omega _{2}t})\,x\,(A\psi _{1}e^{-i\omega _{1}t}+B\psi _{2}e^{-i\omega _{2}t})$
          $=A^{2}\int \!dx\,\psi _{1}^{2}\,x+B^{2}\int \!dx\,\psi _{2}^{2}\,x+AB(e^{i(\omega _{1}-\omega _{2})t}+e^{i(\omega _{2}-\omega _{1})t})\int \!dx\,\psi _{1}x\psi _{2}.$
The first two integrals are the (time-independent) mean positions of a particle associated with $\psi _{1}(x)\,e^{i\omega _{1}t}$ and $\psi _{2}(x)\,e^{i\omega _{2}t},$ respectively. The last term equals
    $2AB\cos(\Delta \omega \,t)\int \!dx\,\psi _{1}x\psi _{2},$
and this tells us that the particle's mean position oscillates with frequency $\Delta \omega =\omega _{2}-\omega _{1}$ and amplitude $2AB\int \!dx\,\psi _{1}x\psi _{2}$ about the sum of the first two terms.
Visit this site to watch the time-dependence of the probability distribution associated with a quantum bouncer that is initially associated with a Gaussian distribution.
NEXT >
# This Quantum World/Implications and applications

< PREVIOUS

# The Schrödinger equation: implications and applications

In this chapter we take a look at some of the implications of the Schrödinger equation
    $i\hbar \,{\frac {\partial \psi }{\partial t}}={\frac {1}{2m}}\left({\frac {\hbar }{i}}{\frac {\partial }{\partial \mathbf {r} }}-\mathbf {A} \right)^{2}\psi +V\psi .$
NEXT >
# This Quantum World/Game

< PREVIOUS

## A quantum game

Here are the rules:
* Two teams play against each other: Andy, Bob, and Charles (the "players") versus the "interrogators".
* Each player is asked either "What is the value of X?" or "What is the value of Y?"
* Only two answers are allowed: +1 or −1.
* Either each player is asked the X question, or one player is asked the X question and the two other players are asked the Y question.
* The players win if the product of their answers is −1 in case only X questions are asked, and if the product of their answers is +1 in case Y questions are asked. Otherwise they lose.
* The players are not allowed to communicate with each other once the questions are asked. Before that, they are permitted to work out a strategy.
Is there a failsafe strategy? Can they make sure that they will win? Stop to ponder the question.
Let us try pre-agreed answers, which we will call X$_A$, X$_B$, X$_C$ and Y$_A$, Y$_B$, Y$_C$. The winning combinations satisfy the following equations:
    $X_{A}Y_{B}Y_{C}=1,\quad Y_{A}X_{B}Y_{C}=1,\quad Y_{A}Y_{B}X_{C}=1,\quad X_{A}X_{B}X_{C}=-1.$
Consider the first three equations. The product of their right-hand sides equals +1. The product of their left-hand sides equals X$_A$X$_B$X$_C$, implying that X$_A$X$_B$X$_C$ = 1. (Remember that the possible values are ±1.) But if X$_A$X$_B$X$_C$ = 1, then the fourth equation X$_A$X$_B$X$_C$ = −1 obviously cannot be satisfied.
    The bottom line: There is no failsafe strategy with pre-agreed answers.
* ↑ Lev Vaidman, "Variations on the theme of the Greenberger-Horne-Zeilinger proof," Foundations of Physics 29, pp. 615-30, 1999.
NEXT >
# XQuery/REST interface definition

REST interfaces, in particular the URL language used to invoke services, has no equivalent to SOAP's WSDL. This example looks a creating a simple XML schema for defining such an interface, and using an XQuery script to create a generic interface to the site based on the interface definition.

## Example REST definition

Here is a somewhat partial definition of the del.icio.us interface using a home-made schema. Parameters are defined using unique local names for each parameter, and then the services supported by the interface are defined using templates with curly braces delimiting the names of parameters, to be replaced by their actual values.

## Generate the interface

An XQuery script takes one parameter called uri, the uri of the XML interface description. The script creates a generic interface based on this definition, regenerating the service urls when values are changed in the form and the form refreshed.
del.icio.us interface

## Discussion
### Architecture

The script uses a common layered architecture in which low level functions operate on the base data model, and these functions are in turn used by functions which generate the user interface. Finally class and id hooks in the generated XHTML link with CSS to style the page. Determining how many layers to use and how the layers should interface is a central design decision in XQuery application, as it is in other technologies. Several alternatives are worth considering: the script generates an intermediate XML structure which is transformed server- or client-side with XSLT; the script generates an XForm in place of the HTML form; the whole task is handled client-side with JavaScript; client-side AJAX interfaces with a base XQuery script. Handling this design space is one of the challenges of web development.

### Cache busting

For scripts running inside a proxy server,as these scripts are on the UWE server, repeated access to the same url in the doc function will return the cached file. To break the cache, a random number is added to the URL.

### Global Variables

The script uses variable declarations to define some global variables used in the script functions. Global variables feel like a reversion to Fortran COMMON and similar horrors, except that these are all constant once defined. Nonetheless, the dependence on these variables is not explicit. An alternative would be to explicitly pass this data down through the functions. An alternative script using this style, passing a single node which composed the data into a single 'object', executes several times slower, is more verbose and arguably no more understandable.

### Recursion

Replacement of the multiple parameters in a template is a recursive function, successively replacing each parameter throughout the template in turn.

## Other interface

* Flickr
# XQuery/Registered Functions
## Motivation

You want a list of all functions or all modules and their functions.

## Method

There are two functions that we can use to get a list of functions in the current run-time system:
The first function returns all registered functions, the second returns all registered functions for a given module.

## List all registered functions

The following XQuery creates a list of all the XQuery functions in alphabetical order. The output will depend on the modules configured in the installation.
Run
Note that if there is no namespace prefix, the function is an XPath library function. (or the math module which also appears without a prefix ??)

### Listing all functions by module

A more useful format would be to list functions by module:
Run
Sample Output:
# XQuery/Returning the Longest String
## Motivation

You want to find the longest string(s) in a sequence of strings.

## Sample Program

Execute

## Discussion

This XQuery creates a local function that takes zero or more strings:
$string-seq as xs:string*
and returns one or more strings:
as xs:string+
It uses the max XPath function that looks at a sequence of values and returns the highest.
Note that if there are several strings in the input set that each have the same max length, it will return all strings of max length.
If you only want the first returned, add "[1]" to the return expression:
return $string-seq[string-length(.) = $max][1]