ipfs-search
latest

Contents:

  • Project information
  • Guides
  • API
  • Data types and application architecture
  • Towards distributed search for the IPFS
    • Distributed search
      • What does it mean for search to be distributed?
        • Distributed search engines
        • IPFS search engines
        • Source
      • The path to distributed search
        • Source
    • Distributed and assistive technologies
    • Current implementations
    • Internet-facing demo
    • Making ipfs-search distributed
ipfs-search
  • »
  • Towards distributed search for the IPFS »
  • Distributed search
  • Edit on GitHub

Distributed search

Our ambition is to make ipfs-search into a truly distributed search engine. We are grateful To Nina for the work she put in to researching distributed search on behalf of ipfs-search. The following is adapted from the results, first published on Nina’s homepage.

What does it mean for search to be distributed?

Distributed search engines

Sadly most that existed have gone defunct, leaving seekstorm (former FAROO (Propietary, English), Seeks (Open Source, English) and YaCy (Fully distributed).

IPFS search engines

Noetic Noetic searches IPFS content as indexed by conventional search engines. Noetic has had no commits over the past three years.

Trinity Trinity is a more promising but currently unfunctional effort to make a search engine and has had no commits for a year.

ipfssearch.xyz ipfsearch.xyz searches decentrally but requires a pre-compiled database.

dweb.page dweb.page uses IOTA, a permissionless distributed ledger allowing every single user to run their own search engine.

Source

The path to distributed search

Learning paths of technology evolutions which we travelled for getting transforming ideas for dweb search.

Distributed technology

  • Blockchain

  • Hashgraph

  • DAG

  • Holochain

Assistive technologies

  • Zero knowledge proofs

  • Smart contracts

  • Voting

  • Gossip about gossip

Current implementations

  • IPFS

  • BTFS

  • FileCoin

  • The usual peer crawling

Internet-facing demo

  • Current IPFS Search architecture

  • Good enough indexing

Technologies for distributed search

  • IPFS

  • Overlay networks

  • Parsing

  • Distributing the index on an IPFS Cluster

  • Querying the index

  • Yggdrasil

  • Testing

Moonshot for realizing distributed search

  • Provider nodes that wish to participate, parse and index only the files they have added to a dweb (DHT hashes) and that have world file permissions.

  • This local index is put on an (IPFS) cluster.

  • A query can use the distributed index.

  • Initial search functionalities are basic boolean search, to begin with.

  • Settings functionality anticipates tuning.

  • In the future, one can add to the search engine functionality with extensions.

Source

Previous Next

© Copyright 2021, Mathijs de Bruin. Revision c3d96877.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: latest
Versions
latest
readthedocs
docgrouping
Downloads
On Read the Docs
Project Home
Builds