YaCy '#[clientname]#': Index Pack Generator

#(reload)#::#(/reload)# #%env/templates/metas.template%#
#%env/templates/header.template%# #%env/templates/submenuIndexImport.template%#

## YaCy Pack Generator

The local index currently contains #[ucount]# documents, only #[ucount200]# exportable with status code 200 - the remaining are error documents.

#(lurlexport)#::
Index Pack Generator

Set a Category (this goes into the filename)
Index Collection
  the collection name is used as part of the filename to describe the content. Exception: if the collection is "user", then you can name the content with a slug.
Slug - describe the content
(only if collection is "user")
  This will become a part of the filename, spaces will be replaced by "-"; must not be empty; should end with a language description, e.g. "-en"
URL Filter
  .*.* (default) is a catch-all; format: java regex
Search Query -
  *:* (default) is a catch-all; format: :
Export Format
  This JSON is an elasticsearch index dump format and can be bulk-imported to elasticsearch. Here is an example for opensearch, using docker:
  Start docker container of opensearch:
  `docker run --name opensearch -p 9200:9200 -d -e OPENSEARCH_JAVA_OPTS="-Xms2G -Xmx2G" -e discovery.type=single-node -e DISABLE_SECURITY_PLUGIN=true -v $(pwd)/opensearch_data:/usr/share/opensearch/data opensearchproject/opensearch:latest`
  Unblock index creation:
  `curl -X PUT "http://localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d' { "persistent": { "cluster.blocks.create_index": null } }'`
  Create the search index:
  `curl -X PUT "http://localhost:9200/collection1/yacy"`
  Bulk-upload the index file:
  `curl -XPOST "http://localhost:9200/collection1/yacy/_bulk?filter_path=took,errors" -H "Content-Type: application/x-ndjson" --data-binary @yacy_dump_XXX.flatjson`
  Make a search, get 10 results, search in fields text_t, title, description with boosts:
  `curl -X POST "http://localhost:9200/collection1/yacy/_search" -H 'Content-Type: application/json' -d' {"size": 10, "query": {"multi_match": { "query": "one two three", "fields": ["text_t", "title^10", "description^3"], "fuzziness": "AUTO" }}}'` JSON (Rich and full-text Elasticsearch data, one document per line in one flat JSON file)
  XML (Rich and full-text Solr data, one document per line in one large xml file, can be processed with shell tools, can be imported with DATA/PACKS/load/)
  XML (RSS)

::
Export to file #[exportfile]# is running .. #[urlcount]# Documents so far
:: #(/lurlexport)# #(lurlexportfinished)#::
Finished export of #[urlcount]# Documents to file #[exportfile]#
Import this file by moving it to DATA/PACKS/load
:: #(/lurlexportfinished)# #(lurlexporterror)#::
Export to file #[exportfile]# failed: #[exportfailmsg]#
:: #(/lurlexporterror)#
Pack List

Pack | Process | Size (KB)
#{packs}#
#[file]# | #[type]# | #[size]#
#{/packs}#

#%env/templates/footer.template%#