Organize Interwiki and Sites in MediaWiki for best efficiency
Tags: mediawiki, interwikilink, sitestable, lua
Published: 2020-04-25 13:14:23

Within the scope of MediaWiki, InterWiki links and the `sites` table seems to have little in common, on the contrary in this article I will explain how a unified management of this two features can simplify working with both and clear the readibility.

## Anatomy of Interwiki
Interwiki links are special links that allows to reach a page on a MediaWiki installation on any kind, such a custom one, or any Wikipedia site.
This functionality works an a MediaWiki installation out of the box as several InterWiki links are defined by default. So you can immediately use links like these
```wikitext
[[wikipedia:Jean-Luc Picard|Jean-Luc Picard on English Wikipedia]]
[[memoryalpha:Jean-Luc Picard|Jean-Luc Picard on English ''Memory Alpha'']]
```
to call pages on a foreign wiki – in this specific example, the English Wikipedia and the English Memory Alpha wikis – without using external links.

You might notice something *strange* in the above wikilinks: the prefixes followed by the colon symbol and then a title of the page.
These prefixes instruct the MediaWiki installation to link a page in another site running MediaWiki, it has several advantages against a plain HTML external link, readability and the lack of "external website" warning on save, among the others.
A complete explanation on interwiki links is [on the manual](https://www.mediawiki.org/wiki/Manual:Interwiki), for those interested. For the moment be content to know that the magic is done by the information matched with the `iw_prefix` in the [`interwiki` table](https://www.mediawiki.org/wiki/Manual:Interwiki_table) in the database.
Keep this information in mind for later in this article.

## SiteLinks and `site` table
The `site` database table is of paramount importance for another feature called "SiteLinks". While you might be familiar with SiteLinks as a feature of WikiData, linking any *item* to pages on sites of the Wikipedia family, in general terms those are special links coming from *any* MediaWiki site with [Wikibase](https://wikiba.se) installed to any other MediaWiki installation.

SiteLinks needs first to be [enabled and configured](https://www.mediawiki.org/wiki/Wikibase/Installation#Enable_Sitelinks) in the WikiBase *Repo* with a string like:
```php
$wgWBRepoSettings['siteLinkGroups'] = [ 'wikipedia', 'mywikigroup', 'anothergroup' ];
```
The "groups" mentioned in the above configuration are defined in the `sites` table we discussed before: each groups contains one or more site with a unique identifier named `site_global_key`

So, now that we know the two characters, we can proceed with their story.

## Organise things
If you had a look at the database titles you might have noticed some common properties:
* `iw_prefix` and `globalid` both are kind of *unique identifiers* for wiki
* `iw_url` and `page_path` both points to the basic page address on the wikis

There is a clear advantage in using a unified notation for `iw_prefix` and `globalid` so the same prefix can be used both in InterWiki and SiteLinks.
So I resolved to create a kind of *master table* of sites I wanted to use in my wiki, that looked like this:

| globalid / iw_prefix | group     | page_path / iw_url                      | iw_api                                  | file_path                          | languagecode | description|
| -------------------- | --------- | --------------------------------------- | --------------------------------------- | ---------------------------------- | ------------ | ---------------------- |
| enma                 | wikia     | https://memory-alpha.fandom.com/wiki/$1 | https://memory-alpha.fandom.com/api.php | https://memory-alpha.fandom.com/$1 | en           | Memory Alpha (inglese) |
| enwiki               | wikipedia | https://en.wikipedia.org/wiki/$1        | https://en.wikipedia.org/w/api.php      | https://en.wikipedia.org/w/$1      | en           | Wikipedia (inglese)    |

I then wrote a small utility to extract data from this table (which I organized in a `|`-separated CSV file for simplicity) and create three thing:
1. a group of SQL statement to insert interwiki prefixes as [explained in the manual](https://www.mediawiki.org/wiki/Manual:Interwiki#Single_line)
2. a XML file [to be imported](https://www.mediawiki.org/wiki/Manual:Sites_table#Adding_a_new_site) in the `sites` table with the maintenance script `importSites.php`. This is not the only way to populate the `sites` table, but in my experience I found it to be the easiest one
3. a code fragment in LUA to be used in [custom modules](https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua)

Below you can see the result from the example above.
The query for interwiki:
```sql
INSERT INTO interwiki (iw_prefix, iw_url, iw_api, iw_wikiid, iw_local, iw_trans) VALUES ('enma', 'https://memory-alpha.fandom.com/wiki/$1', 'https://memory-alpha.fandom.com/api.php', '', 0, 0);
INSERT INTO interwiki (iw_prefix, iw_url, iw_api, iw_wikiid, iw_local, iw_trans) VALUES ('enwiki', 'https://en.wikipedia.org/wiki/$1', 'https://en.wikipedia.org/w/api.php', '', 0, 0);
```

The XML to be imported in `sites`
```xml
<?xml version="1.0"?>
<sites xmlns="http://www.mediawiki.org/xml/sitelist-1.0/" version="1.0">
  <site type="mediawiki">
    <globalid>enma</globalid>
    <group>wikia</group>
    <path type="page_path">https://memory-alpha.fandom.com/wiki/$1</path>
    <path type="file_path">https://memory-alpha.fandom.com/$1</path>
    <languagecode>en</languagecode>
  </site>
  <site type="mediawiki">
    <globalid>enwiki</globalid>
    <group>wikipedia</group>
    <path type="page_path">https://en.wikipedia.org/wiki/$1</path>
    <path type="file_path">https://en.wikipedia.org/w/$1</path>
    <languagecode>en</languagecode>
  </site>
</sites>
```

Finally, the variable definition in LUA code:
```lua
local VariableName = {	
	enma = 'Memory Alpha (inglese)',
	enwiki = 'Wikipedia inglese'	
}
```

## Now make them work together
The source of information for the wiki pages is an *entity* in WikiBase: the raw sitelinks data for the entity are:
```json
{
  "entities": {
    "Q27": {
      
      […]
            
      "sitelinks": {
        "enma": {
          "site": "enma",
          "title": "Jean-Luc Picard",
          "badges": [],
          "url": "https://memory-alpha.fandom.com/wiki/Jean-Luc_Picard"
        },
        "enwiki": {
          "site": "enwiki",
          "title": "Jean-Luc Picard",
          "badges": [],
          "url": "https://en.wikipedia.org/wiki/Jean-Luc_Picard"
        }
      }
    }
  }
}

```

I then created a function in a custom LUA module:
```lua
function p.SiteLinksInterwiki()	
	local AllLinks = {}
	
	local Item = mw.wikibase.getEntity()
	
	local SiteLinks = Item['sitelinks']
	local Titles = {
		enma = 'Memory Alpha (inglese)',
	        enwiki = 'Wikipedia inglese'
	}

	for _, SiteLink in pairs(SiteLinks) do
		local TitleLabel
		
		TitleLabel = Titles[SiteLink['site']] or SiteLink['site']
	
		AllLinks[#AllLinks + 1] = "* [[:" .. SiteLink['site'] .. ":" .. SiteLink['title'] .. "|''" .. SiteLink['title'] .. "'']], " .. TitleLabel
	end
	
	return table.concat(AllLinks, string.char(10))
end
```
Note this function cycles through all members of `sitelinks` node and uses the `site` field as prefix for the interwiki link and to determine the label to be shown after it.

## 💬Conclusion
While this might seem a small details at first sight, in fact this trick greatly improves consistency and thus readability and code cleanliness.
It also gave me the chance to deepen my knowledge of both mechanisms.