XRPL Deep search platform - Developer preview
Tags: xrpl, xrp, search, datapipeline
Published: 2024-01-30 20:19:00

The [XRPScan](https://xrpscan.com) team is pleased to announce the developer preview of [Deep search platform](https://github.com/xrpscan/platform) back-end service (Platform), a data processing pipeline to store XRPL transactions in a distributed search and analytics platform.

XRP Ledger exploration and reporting tools available today (rippled, clio and various explorer APIs) provide access to ledger data based on object's primary key such as ledger index, transaction hash, account address, NFT id, object ids etc.

This project aims to provide deeper search capability such as filtering transactions by source/destination tags, range query over payment amounts, aggregate volumes and much more. This is enabled by indexing all properties of the transactions in an analytics engine.

Platform requires access to a rippled server. Optionally, it can also back-fill older ledgers by connecting to a full-history server.

## Architecture

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9unygxzy7974k6v76vqq.png)

## Installing platform

Platform is known to run on Linux and macOS. Please follow the [installation guide](https://github.com/xrpscan/platform?tab=readme-ov-file#installation) to set it up on CentOS.

## Running the service

Run `bin/platform-server` to start the service. It will subscribe to your rippled server and start indexing transactions, ledgers and validations.

## Back-filling data

```
./bin/platform-cli backfill -verbose -from 82000000 -to 82999999
```
Optionally, a custom full-history server URL may be provided via the command line
```
./bin/platform-cli backfill -verbose -from 82000000 -to 82999999 -server wss://xrplcluster.com
```

## Searching transactions

A front-end for searching transactions would be available in the future. In this developer preview, transactions can be searched by connecting to Elastisearch directly.

```
curl -k -u elastic:$ELASTICSEARCH_PASSWORD \
-H 'Content-type: application/json' \
-XPOST 'https://localhost:9200/platform.transactions/_search' \
-d '{"query":{"term":{"ctid":"C511CC0400850000" }}}' \
| jq .hits.hits
```
Response:
```
[
  {
    "_index": "platform.transactions-85m",
    "_id": "E7B699BA359C4E9E48D64DE17D2914BF97C922B32BF03493A1B95C48BC51CB8D",
    "_score": 15.314049,
    "_source": {
      "Account": "rMdG3ju8pgyVh29ELPWaDuA74CpWW6Fxns",
      "Amount": {
        "currency": "XRP",
        "value": 17805999000
      },
      "Destination": "rsyncTM4ojq33ZVJeBmDXvvJibVftJ2sPy",
      "Fee": 10,
      "Sequence": 3365263,
      "SigningPubKey": "0255EECA852E7C26C0219F0792D1229F1147366D4C936FF3ED83AC32354F6F8EF3",
      "TransactionType": "Payment",
      "TxnSignature": "304402203C4CF26B45FFB8DBB357421AD5E75C38C3CD3D5E290D2E3B0EA9F06548970904022063841E568F02BAE096775017EC7FFD9A12BEE2CD5858650D0870ECF491DD6B94",
      "ctid": "C511CC0400850000",
      "date": 757688782,
      "hash": "E7B699BA359C4E9E48D64DE17D2914BF97C922B32BF03493A1B95C48BC51CB8D",
      "ledger_index": 85052420,
      "meta": {
        "TransactionIndex": 133,
        "TransactionResult": "tesSUCCESS",
        "delivered_amount": {
          "currency": "XRP",
          "value": 17805999000
        }
      },
      "validated": true
    }
  }
]
```

## Share your feedback

We invite you to run Deep search platform and report your observations, bugs and feature requests. The project is available at [https://github.com/xrpscan/platform](https://github.com/xrpscan/platform)