plugin:aichat
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
plugin:aichat [2024-03-13 23:50] – version upped andi | plugin:aichat [2024-05-16 23:50] (current) – version upped andi | ||
---|---|---|---|
Line 6: | Line 6: | ||
email : dokuwiki@cosmocode.de | email : dokuwiki@cosmocode.de | ||
type : action, helper, syntax, CLI | type : action, helper, syntax, CLI | ||
- | lastupdate : 2024-03-13 | + | lastupdate : 2024-05-16 |
compatible : Jack Jackrum, Kaos | compatible : Jack Jackrum, Kaos | ||
depends | depends | ||
conflicts | conflicts | ||
similar | similar | ||
- | tags : !experimental, | + | tags : !experimental, |
downloadurl: | downloadurl: | ||
Line 17: | Line 17: | ||
sourcerepo : https:// | sourcerepo : https:// | ||
donationurl: | donationurl: | ||
+ | |||
+ | updatemessage: | ||
screenshot_img : plugin: | screenshot_img : plugin: | ||
Line 23: | Line 25: | ||
[[https:// | [[https:// | ||
- | This plugin adds the ability to use a " | + | This plugin adds the ability to use a Large Language Model (LLM) often called |
You can read an [[https:// | You can read an [[https:// | ||
Line 31: | Line 33: | ||
:!: This plugin is experimental. Implementation details may change and require manual intervention on future updates. | :!: This plugin is experimental. Implementation details may change and require manual intervention on future updates. | ||
- | :!: The use of this plugin creates | + | :!: The use of this plugin creates API costs. Be sure to be aware of the pricing details |
:!: This plugin requires PHP 8.1 or higher. Installing it on systems with lower versions will " | :!: This plugin requires PHP 8.1 or higher. Installing it on systems with lower versions will " | ||
Line 47: | Line 49: | ||
===== Configuration ===== | ===== Configuration ===== | ||
+ | If you need help with installing, configuring and fine tuning this plugin, feel free to [[https:// | ||
- | ==== OpenAI ==== | + | Use the [[config|Configuration Manager]] to adjust the settings described below. |
- | You need to create an [[https:// | ||
- | Configure the key in the [[plugin: | + | ==== 🧠 Models ==== |
- | You can select the AI model used here as well. Currently GPT-3.5 models with 4k and 16k token limits | + | Models are used to create embeddings (vectors describing your content), rephrasing |
- | ==== Vector Storage ==== | ||
- | The plugin needs to store embedding vectors to do semantic similarity searches. There are multiple options available, you only need to configure one. | + | * [[https:// |
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
- | === SQLite Storage === | + | You can use different models for embeddings, rephrasing and chat. When changing the embedding model you need to rebuild the vector storage (see below). The rephrasing model is used to interpret a given question |
- | This is the default mechanism. It uses the [[sqlite]] plugin to store vectors in a SQLite database. The more wiki pages you have, the longer does it take to search through all available embeddings. To mitigate this, this storage mechanism uses a clustering approach, separating | + | An overview over prices, properties and a short description of each model can be seen when running |
- | Clusters are automatically created when the embedding index is built the first time. However, when your wiki grows a lot, topics change drastically or you add a new language, the initial clusters may not longer be a good fit. In that case you should run the '' | + | Which model is the best for your use case depends on various factors and requires some experimentation. Contact CosmCode |
- | === PineCone | + | ==== 📥 Vector |
- | This mechanism uses the [[https:// | + | The plugin needs to store embedding vectors to do semantic similarity searches. There are multiple options available, you only need to configure one. |
- | Their free tier should be enough to get you started. | + | The default uses a local SQLite database. This does not need any additional configuration. However, the more wiki pages you have, the longer does it take to search through all available embeddings. To mitigate this, this storage mechanism uses a clustering approach, separating the similar page chunks into clusters and then only search the nearest cluster when looking for similar chunks. |
- | * Sign up at [[https:// | + | Clusters are automatically created when the embedding index is built the first time. However, when your wiki grows a lot, topics change drastically or you add a new language, the initial clusters may not longer be a good fit. In that case you should run the '' |
- | * Create | + | |
- | * Use a dimension of '' | + | |
- | * The list of indexes will give you the '' | + | |
- | * Under API Keys you find the key to use for the '' | + | |
+ | The other storage options are dedicated vector storage solutions, that each need their own configuration: | ||
- | Once you set the '' | + | * [[https:// |
+ | * [[https:// | ||
+ | * [[https:// | ||
- | === Chroma Storage | + | ==== Finetuning ==== |
- | [[https:// | + | The plugin offers |
- | Once you configure a '' | + | * chunkSize - The maximum size (in tokens) of chunks your wiki pages will be split into. |
- | + | * similarityThreshold | |
- | + | * contextChunks - the maximum number of chunks to use as context when answering question. More chunks also mean larger context and higher cost! | |
- | === Qdrant Storage === | + | * rephraseHistory - how many question/ |
- | + | * chatHistory - how many question/ | |
- | [[https:// | + | |
- | + | ||
- | Alternatively there is a free hosted trial available: | + | |
- | + | ||
- | * Sign up at https:// | + | |
- | * Create a cluster | + | |
- | * keep all sliders to the left for the free tier | + | |
- | * Create an API key for your cluster | + | |
- | * the curl snippet displayed shows you your baseurl and apikey | + | |
- | + | ||
- | Once you configure a '' | + | |
Line 135: | Line 129: | ||
All chat conversations can optionally be logged to DokuWiki' | All chat conversations can optionally be logged to DokuWiki' | ||
- | Access to the chat functionality can be restricted to users and groups | + | Access to the chat functionality can be restricted to users and groups |
===== Initial Setup ===== | ===== Initial Setup ===== | ||
- | Once the plugin has been configured, the embeddings | + | Once the plugin has been configured, the vector |
Line 147: | Line 141: | ||
Updates to the storage can be done by rerunning the command. | Updates to the storage can be done by rerunning the command. | ||
- | Depending on the size of your wiki this can take a while. Whenever you change | + | Depending on the size of your wiki this can take a while. Whenever you change |
php bin/ | php bin/ | ||
Line 226: | Line 220: | ||
php bin/ | php bin/ | ||
- | Shows a few statistics on the vector store. | + | Shows a few statistics on the currently configured |
php bin/ | php bin/ | ||
Line 238: | Line 232: | ||
* Does this work on multilingual content? | * Does this work on multilingual content? | ||
- | * It seem to work just fine. ChatGPT | + | * It seem to work just fine. Most LLMs can work with many languages and will automatically match the user's question language to content in the same language. |
* Can I adjust the prompts used? | * Can I adjust the prompts used? | ||
- | * Yes you'll find them in the '' | + | * Yes you'll find them in the '' |
* Will the chat bot hallucinate? | * Will the chat bot hallucinate? | ||
* Yes. Even though the prompt asks the model not to, it has a tendency to state things with confidence that are not entirely true. Remember this is not real intelligence or understanding. It is just clever text prediction. However as long as the question covers a topic that does have an answer in your wiki, the results are generally pretty good. | * Yes. Even though the prompt asks the model not to, it has a tendency to state things with confidence that are not entirely true. Remember this is not real intelligence or understanding. It is just clever text prediction. However as long as the question covers a topic that does have an answer in your wiki, the results are generally pretty good. | ||
* Will the bot answer questions that are unrelated to the wiki contents? | * Will the bot answer questions that are unrelated to the wiki contents? | ||
- | * That might happen. No answers will be given if no fitting sources can be found at all -- a similarity cut-off is used. But often sources are found that are just slightly above the cut-off but do not answer the question. The bot will then fall back to its ChatGPT | + | * That might happen. No answers will be given if no fitting sources can be found at all -- check the '' |
- | * Can I use a different model? Like Bard? Or even running my own? | + | * Can I use a different model? Or even running my own? |
- | * Models can be added relatively easy (as long as there is an embedding mechanism). Again, contact us and we can work something out. | + | * Models can be added relatively easy. Again, contact us and we can work something out. |
+ | * The bot seems to never find any sources? | ||
+ | * Check the '' | ||
===== Development ===== | ===== Development ===== | ||
- | Developers could use this plugin in their own plugins. Via the helper component you get access to an implementation of AbstractModel which will give you a Client to interact with the current model provider | + | Developers could use this plugin in their own plugins. Via the helper component you get access to an implementation of AbstractModel which will give you a Client to interact with the current model provider. You also get access to the Embedding object which lets you do similarity searches on the wiki content. |
+ | |||
+ | ===== See also ===== | ||
+ | |||
+ | Forum thread: [[https:// | ||
plugin/aichat.1710370208.txt.gz · Last modified: 2024-03-13 23:50 by andi