YaCy '#[clientname]#': LLM Selection

#%env/templates/metas.template%#
#%env/templates/header.template%# #%env/templates/submenuAI.template%#

## LLM Selection

Here you can pick models from a LLM model service to select them as production model. In the "Production Models Matrix" you can then assign each selected model a function inside YaCy

Install your local LLM service! You need either a local ollama or LM Studio instance running on your local host or inside the intranet.

Service Selection

service
  This makes a preset to the Hoststub value
hoststub
  you can probably leave this to the default value
api_key
  (not required for Ollama or LMStudio)
max_tokens
  You must set the Context Length in the LLM service to fit to your selected max_tokens; in Ollama you find a Context Length slider in the settings

Model Downloads
Production Models Matrix

service | model | hoststub | api_key | max_tokens | search-answers This model creates answers for search requests | chat This model is used in the chat interface and as default for the RAG proxy | translation This model can be used to make translations of the web UI | classification This model is used to classify prompts to find out what they demand | search-query This model produces search queries to YaCy search from prompts in RAG or chat | qa-pairs This model can be used to produce query-answer pairs which enhance search from chat prompts | tldr-shortener This model is used to make summaries from web content | thinking we detect thinking only to be able to suppress thinking. thinking is not used in YaCy | tooling tooling is required for agentic abilities. | vision this enables image recognition in the chat | format this is required for classification | Actions
#{productionmodels}#
#[service]# | #[model]# | #[hoststub]# | #[api_key]# | #[max_tokens]# |  |  |  |  |  |  |  | #[thinking]# | #[tooling]# | #[vision]# | #[format]# |
#{/productionmodels}#

#%env/templates/footer.template%#