Indices

Elasticsearch index mapping and settings

To be used in the Create index API:

$ curl -d <index.json> -X PUT http://localhost:9200/<index-name>

Example entries

Examples of real-life crawled content are available for a file and a directory.

Reindexing

  1. Stop crawler.

$ systemctl stop ipfs-crawler
  1. Create snapshot to allow for rollback:

PUT
/_snapshot/ipfs/snapshot_v<old>
  1. Create new index:

PUT /ipfs_v<new>
<<< index-json >>>
  1. Reindex old to new:

POST /_reindex
{
  "source": {
    "index": "ipfs_v<old>"
  },
  "dest": {
    "index": "ipfs_v<new>"
  }
}

(Go fetch some coffee for this one.)

  1. Remove old alias, create new alias:

POST /_aliases
{
    "actions" : [
        { "remove" : { "index" : "ipfs_v<old>", "alias" : "ipfs" } },
        { "add" : { "index" : "ipfs_v<new>", "alias" : "ipfs" } }
    ]
}
  1. Restart crawler:

$ systemctl start ipfs-crawler
  1. Remove old index (after verifying everything is ok):

DELETE /ipv_v<old>