YaCy '#[clientname]#': Index Sources & Targets

#%env/templates/metas.template%#
#%env/templates/header.template%# #%env/templates/submenuIndexControl.template%#

## Index Sources & Targets

YaCy supports multiple index storage locations. As an internal indexing database a deep-embedded multi-core Solr is used and it is possible to attach also a remote Solr.

Solr Search Index Solr stores the main search index. It is the home of two cores, the default 'collection1' core for documents and the 'webgraph' core for a web structure graph. Detailed information about the used Solr fields can be edited in the Schema Editor.

Lazy Value Initialization
  If checked, only non-zero values and non-empty strings are written to Solr fields.
Use deep-embedded local Solr
  This will write the YaCy-embedded Solr index which stored within the YaCy DATA directory.
  The Solr native search interface is accessible at
  /solr/select?q=*:*&start=0&rows=3&core=collection1 for the default search index (core: collection1) and at
  /solr/select?q=*:*&start=0&rows=3&core=webgraph for the webgraph core.
  If you switch off this index, a remote Solr must be activated.
Use remote Solr server(s)
  It's easy to attach an external Solr to YaCy. This external Solr can be used instead the internal Solr. It can also be used additionally to the internal Solr, then both Solr indexes are mirrored.
Allow self-signed certificates
  Tick this when the remote Solr server is password protected and is requested over HTTPS but provides only a self-signed certificate (not a validated one by an official Certificate Authority). The Solr URL could be for example something like https://user:password@localhost:8984/solr.
#(table)#::
  Solr Hosts

  Solr Host Administration Interface | Index Size
  #{list}#
  #[url]# | #[size]#
  #{/list}#

#(/table)#
  Solr URL(s)
  #[solr.indexing.url]#
  You can set one or more Solr targets here which are accessed as a shard. For several targets, list them using a ',' (comma) as separator. The set of remote targets are used as shard of a complete index. The host part of the url is used as key for a hash function which selects one of the shards (one of your remote servers). When a search request is made, all servers are accessed synchronously and the result is combined.
  Sharding Method
  write-enabled (if unchecked, the remote server(s) will only be used as search peers)

Web Structure Index The web structure index is used for host browsing (to discover the internal file/folder structure), ranking (counting the number of references) and file search (there are about fourty times more links from loaded pages as in documents of the main search index).

  use citation reference index (lightweight and fast)
  use webgraph search index (rich information in second Solr core)

Peer-to-Peer Operation The 'RWI' (Reverse Word Index) is necessary for index transmission in distributed mode. For portal or intranet mode this must be switched off.

  support peer-to-peer index transmission (DHT RWI index)
Block known error URLs in DHT
  Reject URLs/RWIs with known errors from peers. Disable to opt out.
Retry after (days)
  for temporary errors; permanent errors stay blocked.
Permanent error statuses
  comma-separated (default: 404,410,-1; -1=DNS/network errors)

#%env/templates/footer.template%#