Saravanan666/pypi-org huggingface dataset:
Package: llmkira
🍩 部署文档&🧀 开发文档&📝 English Readme&🤝 贡献必看Look for English README? Clickhere. We also have English
documentationhere.部署遇到问题？提交 Issue 帮助我们提升可用性此项目为自部署，实用可扩展的机器人核心，以FunctionCallToolCall为核心，支持多种消息平台。采用消息队列，很好处理函数请求，支持繁杂的插件和功能设计。良好支持文件系统。支持多种模型源，支持跨平台消息转发。Demo与之前的项目不同的是，此项目尝试基于消息平台复刻 ChatGpt 的插件系统，实现部分或更进一步的功能。因为 func call 为 feature,所以只支持 Openai 类型的 api, 不打算支持没有 func call 的 LLM📦 Feature🍪 完善的插件开发生态，采用经典设计，通过pip安装即可使用📝 消息系统，不限时间，不限发送端，定义发送接收者，逻辑完全解耦📎 路由消息，自定义消息路由，以路由决定运作方式📬 公共开放限额/私人自配置后端/代理Token认证，提供灵活可扩展的鉴权开发方案🍾 支持中间件拦截开发，开发扩展即可操作流程前后数据🎵 细化的统计系统，轻松统计使用情况🍰 支持插件人在回路验证，可鉴权，可设置插件黑名单📦 完善标准的文件交互支持，上传/下载文件🍖 支持个人单独配置环境密钥，为插件提供个人的私有环境变量🍟 支持大语言模型增量支持，支持多平台扩展，继承标准类即可适配🍔 同时支持FunctionCallToolCall特性，根据模型动态构建需要的函数类🧀 部分插件预览Sticker ConverterTimer FuncTranslate Func🧀 认证系统介绍我们采用的认证系统称为Service Provider，即服务提供商，它的作用是为每个发送者分配 Endpoint/Key/Model ，用于鉴权。
拥有一个token作为绑定的 OpenKey。程序会调用设定的Service Provider读取私有 Key/配置 Token 来获取鉴权信息。认证组件和后端均需要自行实现。🎬 平台支持平台支持情况文件系统备注Telegram✅✅Discord✅✅Kook✅✅不支持被回复启动Slack✅✅不支持被回复启动QQ❌Wechat❌Twitter❌Matrix❌IRC❌...创建Issue/PR📦 快速开始阅读🧀 部署文档获得更多信息。请提前用pdm run python3 start_sender.pypdm run python3 start_receiver.py测试是否能正常运行。性能指标测试(Until 2023/11/1)注意，不包括pm2，redis，rabbitmq，mongodb，docker等服务的内存占用。进程内存均值测算命令clientreceiver120.202MBpython3 -m memray run --live start_receiver.pytelegramsender83.375MBpython3 -m memray run --live start_sender.pytelegram🥣 DockerBuild Hub:sudoskys/llmbot自动 Docker/Docker-compose安装如果你在使用一台崭新的服务器，你可以使用下面的Shell来尝试自动安装本项目。此脚本会自动使用 Docker 方法安装所需服务并映射端口，如果您已经部署了redis，rabbitmq，mongodb。请自行修改docker-compose.yml文件。curl-sSLhttps://raw.githubusercontent.com/LLMKira/Openaibot/main/deploy.sh|bash手动 Docker-compose安装gitclonehttps://github.com/LlmKira/Openaibot.gitcdOpenaibot
cp.env.exp.env&&nano.env
docker-compose-fdocker-compose.ymlup-d更新镜像使用docker-compose pull。在 docker 中查看 Shell，使用docker exec -it llmbot /bin/bash，输入exit退出。🍔 Shell人工使用Pm2启动，需要自行安装redis，rabbitmq，mongodb。gitclonehttps://github.com/LlmKira/Openaibot.git
pipinstallpdmcdOpenaibot
pdminstall-Gbot
cp.env.exp.env&&nano.env
aptinstallnpm-y&&npminstallpm2&&pm2startpm2.json
pm2monit重启程序使用pm2 restart pm2.json。推荐使用pdm进行依赖管理，因为我们使用了pydantic^1.9.0，为了防止出现版本冲突，我们使用了pdm进行依赖管理。🍪 Slash Commandclear-删除聊天记录help-显示文档
chat-对话
task-启用函数以对话
ask-禁止函数以对话
tool-列出所有函数
set_endpoint-设置私有key和endpoint
clear_endpoint-清除私有key和endpoint
auth-授权一个函数
env-函数环境变量
token-绑定令牌
token_clear-清除令牌绑定
func_ban-禁用一个函数
func_unban-解禁一个函数bind-绑定消息源
unbind-解绑消息源💻 如何开发插件？插件开发文档请参考plugins目录下的示例插件和🧀 插件开发文档🤝 We need your help!This is a long term project and we started the development of the LLM APP very early!We applied a plugin-like system and search online before GPT3 OpenaiApi was released(davinci-003)After many iterations, we have worked hard to make this project more standardized, generic, and open.We can't do it on our own at the moment:We need help with the documentationWeb UIFeel free to submit a Pull Request or discuss, we'd love to receive your contribution!📜 告知此项目与 Openai 官方无关，全称为 OpenAiBot，表示开放人工智能机器人，并不表示为 Openai 所属机器人。如果您所在辖区禁止使用 Openai 服务，请勿使用此项目。
Package: llm-kit
LLM KitA pythonic way to call llms & implement llm algorithms.Free software: MIT licenseDocumentation:https://llm-kit.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2023-05-10)First release on PyPI.
Package: llm-labeling-ui
LLM Labeling UIAboutWARNING:This software is mainly developed according to my personal habits and is still under development. We are not responsible for any data loss that may occur during your use.LLM Labeling UI is a project fork fromChatbot UI, and made the following modifications to make it more suitable for large language model data labeling tasks.The backend code is implemented in python, the frontend code is precompiled, so it can run without a nodejs environmentThe Chatbot UI uses localStorage to save data, with a size limit of 5MB, the LLM Labeling UI can load local data when starting the service, with no size limitWeb interaction:Browse data in pages, search by keywords, filter by messages count.Directly modify/delete model's response results.Split long conversations into multiple conversationsA confirmation button has been added before deleting the conversation messageDisplay the number of messages and token length in the current conversationAllow modify system prompt during the dialogueReplace string in current conversationUsefulcommand line toolsto help you clean/manage your data, such as language cleaning, duplicate removal, embedding cluster, etc.Quick Startpipinstallllm-labeling-ui1. Provide OpenAI API KeyYou can provide openai api key before start server or configure it later in the web page.exportOPENAI_API_KEY=YOUR_KEYexportOPENAI_ORGANIZATION=YOUR_ORG2. Start Serverllm-labeling-uiserverstart--datachatbot-ui-v4-format-history.json--tokenizermeta-llama/Llama-2-7b--data: Chatbot-UI-v4 format, here is anexample. Before the service starts, achatbot-ui-v4-format-history.sqlitefile will be created based onchatbot-ui-v4-format-history.json. All your modifications on the page will be saved into the sqlite file. If thechatbot-ui-v4-format-history.sqlitefile already exists, it will be automatically read.--tokenizeris used to display how many tokens the current conversation on the webpage contains. Please note that this is not the token consumed by calling the openai api.Command Line Toolscluster: Cluster operations, such as create embedding, run cluster, semantic deduplication, etc.conversation: Conversation operations, such as remove prefix, remove deduplication, etctag: Add tags to you data, such as lang classification(en,zh..), traditional or simplified chinese classification, etc.User--helpto see more details, such as:llm-labeling-uicluster--help

Usage:llm-labeling-uicluster[OPTIONS]COMMAND[ARGS]...

╭─Options────────────────────────────────────────────────────────────────────╮
│--helpShowthismessageandexit.│
╰──────────────────────────────────────────────────────────────────────────────╯
╭─Commands───────────────────────────────────────────────────────────────────╮
│create-embeddingCreateembedding│
│dedupDeleteredundantdatainthesameclusteringresult│
│accordingtocertainstrategies.│|prune-embeddingRemoveembeddingnotexistsindb|│runDBSCANembeddingcluster│
│viewViewclusterresult│
╰──────────────────────────────────────────────────────────────────────────────
Package: llmlib
Portability lib for OpenAI, Hugging Face, and FastChat local model APIsThis library was written for examples in a book I wroteYou can read my book free online or can be purchased:Safe For Humans AI - A "humans-first" approach to designing and building AI systemsSupported APIsget_completion(prompt, max_tokens=64)create_local_embeddings_files_in_dir(path_to_document_files_dir)query_local_embeddings(query)These APIs are implemented separately in two Python classes:OpenAiWrapperHuggingFaceAiWrapperSupported APIs and ModelsCurrently this compatibiolity library only works with selected OpenAI APIs and the small Hugging Face modelfacebook/opt-iml-1.3bthat can be run locally on a laptop with no GPU and only 8G of RAM.Run the example programs:python test_completion_openai.py
python test_local_index_openai.py
python test_completion_hf.py
python test_local_index_hf.py
Package: llmlink
LLMLinkLLMLink is a Python SDK designed to simplify the creation of chatbot-based applications using Gradio. The SDK offers an efficient way to interact with Language Learning Models (LLMs) and deploy Gradio applications around these models.ContentsCapabilitiesInstallationQuickstart/UsageCapabilitiesLLMLink's capabilities are all focused around two classes - theModelclass and theAppclass.TheModelclass implements the ability to talk to a specific LLM/chatbot, while theAppclass implements the Gradio application around the specificModel.InstallationTo install LLMLink, you can use pip to install directly from the GitHub repository:pipinstallgit+https://github.com/jacobrenn/llmlinkQuickstart/UsageHere's a simple example of how you can use LLMLink to create an LLM-powered application, and then deploy it:importllmlink.modelasModelimportllmlink.appasAppmodel=Model(openai_api_key='<your OpenAI API key>',model='gpt-3.5-turbo',model_type='chat_openai',memory=True)app=App(model,feedback=True)app.deploy()We are currently still working on building the implementation and documentation around LLMLink, so if you have any specific questions please feel free to reach out to us directly.
Package: llmlite
llmliteA library helps to communicate with all kinds of LLMs consistently.ModelStateSystem PromptNoteChatGPTDone ✅YesLlama-2Done ✅YesCodeLlamaDone ✅YesChatGLM2Done ✅NoChatGLM3WIP ⏳YesBaichuan2Done ✅YesClaude-2RoadMap 📋issue#7FalconRoadMap 📋issue#8StableLMRoadMap 📋issue#11Baichuan2RoadMap 📋issue#34............llmlite also supports different inference backends as below:backendStateNotehuggingfaceDone ✅Support by huggingface pipelinevLLMDone ✅.........How to installpip install llmlite==0.0.9How to useChatfromllmlite.apisimportChatLLM,ChatMessagechat=ChatLLM(model_name_or_path="meta-llama/Llama-2-7b-chat-hf",# requiredtask="text-generation",backend="vllm",)result=chat.completion(messages=[ChatMessage(role="system",content="You're a honest assistant."),ChatMessage(role="user",content="There's a llama in my garden, what should I do?"),])# Output: Oh my goodness, a llama in your garden?! 😱 That's quite a surprise! 😅 As an honest assistant, I must inform you that llamas are not typically known for their gardening skills, so it's possible that the llama in your garden may have wandered there accidentally or is seeking shelter. 🐮 ...llmlitealso supports other parameters liketemperature,max_length,do_sample,top_k,top_pto help control the length, randomness and diversity of the generated text.Seeexamplesfor reference.PromptingYou can usellmliteto help you generate full prompts, for instance:fromllmlite.apisimportChatMessage,LlamaChatmessages=[ChatMessage(role="system",content="You're a honest assistant."),ChatMessage(role="user",content="There's a llama in my garden, what should I do?"),]LlamaChat.prompt(messages)# Output:# 

 
# You're a honest assistant.# 
# There's a llama in my garden, what should I do? 
LoggingSet the env variableLOG_LEVELfor log configuration, default toINFO, others like DEBUG, INFO, WARNING etc..RoadmapAdapter supportQuantizationStreamingContributions🚀 All kinds of contributions are welcomed ! Please followContributing.Contributors🎉 Thanks to all these contributors.
Package: llm-loop
llm-loopOverviewllm-loopis a Python package designed to simplify the process of querying language models (like GPT or similar models) until a response matching a specified pattern is obtained or a maximum number of attempts is reached. This is particularly useful when working with AI models in scenarios where a specific format of response is required.Installationpipinstallllm-loopThis will install the necessary Python packages, includingctransformersand any other dependencies.UsageHere's a basic example of how to usellm-loop:Import the necessary modules:importosfromctransformersimportAutoModelForCausalLM,AutoTokenizerfromllm_loop.mainimportLLMLoopInitialize the model with custom parameters:model_name="YourModelName"model_file="YourModelFileName"start_dir='/path/to/your/model'model_path=f"{start_dir}/{model_file}"llm=AutoModelForCausalLM.from_pretrained(model_name,model_file=model_path,model_type='YourModelType',gpu_layers=YourGPULayers)Create an instance of LLMLoop and query the model:loop=LLMLoop(llm,10)# 10 is the maximum number of attemptsprompt="Your prompt here"pattern=r'Your regex pattern here'response=loop.query_llm(prompt=prompt,pattern=pattern)print("Response:",response)ContributingContributions tollm-loopare welcome! Please feel free to submit pull requests or open issues to suggest improvements or add new features.LicenseMIT.
Package: llm-markov
llm-markovPlugin forLLMadding aMarkov chaingenerating modelInstallationInstall this plugin in the same environment as LLM.llminstallllm-markovUsageThis plugin adds a model calledmarkov. You can execute it like this:llm-mmarkov"The quick brown fox jumps over the lazy dog"My default it will produce 100 words. You can control the number of words with the-o numberoption:llm-mmarkov-o20"The quick brown fox jumps over the lazy dog"A delay of 0.02s is simulated between each token. You can modify this using the-o delayoption - to0to disable it, or some other floating point number of seconds to customize it:llm-mmarkov"The quick brown fox jumps over the lazy dog"-odelay0llm-mmarkov"The quick brown fox jumps over the lazy dog"-odelay0.1-olength20DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cd llm-markov
python3 -m venv venv
source venv/bin/activateNow install the dependencies and test dependencies:pip install -e '.[test]'To run the tests:pytest
Package: llm-mistral
llm-mistralLLMplugin providing access toMistralmodels using the Mistral APIInstallationInstall this plugin in the same environment as LLM:llminstallllm-mistralUsageFirst, obtain an API key forthe Mistral API.Configure the key using thellm keys set mistralcommand:llmkeyssetmistral<paste key here>You can now access the Mistral hosted models. Runllm modelsfor a list.To run a prompt throughmistral-tiny:llm-mmistral-tiny'A sassy name for a pet sasquatch'To start an interactive chat session withmistral-small:llmchat-mmistral-smallChatting with mistral-small
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> three proud names for a pet walrus
1. "Nanuq," the Inuit word for walrus, which symbolizes strength and resilience.
2. "Sir Tuskalot," a playful and regal name that highlights the walrus' distinctive tusks.
3. "Glacier," a name that reflects the walrus' icy Arctic habitat and majestic presence.To use a system prompt withmistral-mediumto explain some code:catexample.py|llm-mmistral-medium-s'explain this code'Model optionsAll three models accept the following options, using-o name valuesyntax:-o temperature 0.7: The sampling temperature, between 0 and 1. Higher increases randomness, lower values are more focused and deterministic.-o top_p 0.1: 0.1 means consider only tokens in the top 10% probability mass. Use this or temperature but not both.-o max_tokens 20: Maximum number of tokens to generate in the completion.-o safe_mode 1: Turns onsafe mode, which adds a system prompt to add guardrails to the model output.-o random_seed 123: Set an integer random seed to generate deterministic results.EmbeddingsThe MistralEmbeddings APIcan be used to generate 1,024 dimensional embeddings for any text.To embed a single string:llmembed-mmistral-embed-c'this is text'This will return a JSON array of 1,024 floating point numbers.TheLLM documentationhas more, including how to embed in bulk and store the results in a SQLite database.SeeLLM now provides tools for working with embeddingsandEmbeddings: What they are and why they matterfor more about embeddings.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-mistral
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:llminstall-e'.[test]'To run the tests:pytest
Package: llm-mpt30b
llm-mpt30bPlugin forLLMadding support for theMPT-30B language model.This plugin usesTheBloke/mpt-30B-GGML. The code was inspired byabacaj/mpt-30B-inference.InstallationInstall this plugin in the same environment as LLM.llminstallllm-mpt30bAfter installing the plugin you will need to download the ~19GB model file. You can do this by running:llmmpt30bdownloadUsageThis plugin adds a model calledmpt30b. You can execute it like this:llm-mmpt30b"Three great names for a pet goat"The alias-m mptworks as well.You can pass the option-o verbose 1to see more verbose output - currently a progress bar showing any additional downloads that are made during execution.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cd llm-mpt30b
python3 -m venv venv
source venv/bin/activateNow install the dependencies and test dependencies:pip install -e '.[test]'To run the tests:pytest
Package: llm-nosrun
llm-nosrunLLMplugin for models hosted byNOS CloudInstallationFirst,install the LLM command-line utility.Now install this plugin in the same environment as LLM.llminstallllm-nosrunConfigurationYou will need an API key from NOS Cloud. You canobtain one here.You can set that as an environment variable calledLLM_NOSRUN_KEY, or add it to thellmset of saved keys using:llmkeyssetnosrunEnter key: <paste key here>UsageTo list available models, run:llmmodelslistYou should see a list that looks something like this:NOSRun: TinyLlama/TinyLlama-1.1B-Chat-v1.0
NOSRun: meta-llama/Llama-2-7b-chat-hf
NOSRun: meta-llama/Llama-2-13b-chat-hf
NOSRun: meta-llama/Llama-2-70b-chat-hf
NOSRun: HuggingFaceH4/zephyr-7b-beta
NOSRun: HuggingFaceH4/tiny-random-LlamaForCausalLM
NOSRun: NousResearch/Yarn-Mistral-7b-128k
NOSRun: mistralai/Mistral-7B-Instruct-v0.2
NOSRun: mistralai/Mixtral-8x7B-Instruct-v0.1
NOSRun: TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ
NOSRun: TheBloke/Mixtral-8x7B-Instruct-v0.1-AWQ
NOSRun: mlabonne/phixtral-2x2_8
NOSRun: mlabonne/phixtral-4x2_8To run a prompt against a model, pass its full model ID to the-moption, like this:llm-mTinyLlama/TinyLlama-1.1B-Chat-v1.0\'Five strident names for a pet walrus'\--system'You love coming up with creative names for pets'You can set a shorter alias for a model using thellm aliasescommand like so:llmaliasessettinyllamaTinyLlama/TinyLlama-1.1B-Chat-v1.0Now you can prompt Llama 2 70B using:catllm_nosrun.py|\llm-mtinyllama-s'explain this code'DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-nosrun
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:pipinstall-e'.[test]'To run the tests:pytest
Package: llmo
LLMO (Elmo)Meaning:ProtectorWorthy to be LovedHelm from God(Most Importantly) A helpful AI programming CLI toolLLMO is the AI pair programming tool that meets you where you are (your terminal) 😉With the"staging area", you can keep files in the context window without the hassle of copying and pasting every time you make changes to your code.FeaturesInteractive Chat: Enjoy real-time, interactive programming assistance in your terminal.Staging Area: Easily add files to the AI's context to update it about your ongoing coding tasks. No need to copy and paste updates.Model Customization: Choose the OpenAI model that fits your needs.Personality: By default, Elmo loves to make bodybuilding references. This can be turned off through a CLI flag or environment variable.InstallationThe recommended way to installllmois throughpipx:pipxinstallllmoDemohttps://github.com/knowsuchagency/llmo/assets/11974795/72419d64-585b-4cd5-a546-2a07de3e6f03Usagellmo--help# you can also use the shorthandlm# You can pass the -s flag if you don't need the full GUI modelm-s"Could you show me an example of valid json?"# Adding files to context# main.py# from utils import add_numbers# result = add_numbers(5, 3)# utils.py# def add_numbers(a, b):#     return a + blm"How can I make add_numbers return a string?"-fmain.py-futils.pyNotesAuthenticationThis application uses theOpenAIAPI.You will need to authenticate with your own API Key.Copying TextTextualruns the terminal in application mode. The means that you can't simply copy content as you normally would.
Initerm2, you can hold down theoptionkey to select text. See the documentation for your terminal emulator for more information.LicenseApache License 2.0DisclaimerThis tool is not officially associated with OpenAI. Always follow OpenAI's use case policy when interacting with their API.
Package: llm-ollama
llm-ollamaLLMplugin providing access to models running on localOllamaserver.InstallationInstall this plugin in the same environment asLLM.llminstallllm-ollamaUsageFirst, ensure that your Ollama server is running and that you have pulled some models. You can useollama listto check what is locally available.The plugin will query the Ollama server for the list of models. You can usellm ollama list-modelsto see the list; it should be the same as output byollama list. All these models will be automatically registered with LLM and made available for prompting and chatting.Assuming you havellama2:latestavailable, you can run a prompt using:llm-mllama2:latest'How much is 2+2?'The plugin automatically creates a short alias for models that have:latestin the name, so the previous command is equivalent to running:llm-mllama2'How much is 2+2?'To start an interactive chat session:llmchat-mllama2Chatting with llama2:latest
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
>Model aliasesThe same Ollama model may be referred by several names with different tags. For example, in the following list, there is a single unique model with three different names:ollamalist
NAMEIDSIZEMODIFIED
stable-code:3baa5ab8afb8621.6GB9hoursago
stable-code:codeaa5ab8afb8621.6GB9secondsago
stable-code:latestaa5ab8afb8621.6GB14secondsagoIn such cases, the plugin will register a single model and create additional aliases. Continuing the previous example, this is what LLM will have:llmmodels
...

Ollama:stable-code:3b(aliases:stable-code:code,stable-code:latest,stable-code)Model optionsAll models accept the following options, using-o name valuesyntax:-o temperature 0.8: The temperature of the model. Increasing the temperature will make the model answer more creatively.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-ollama
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:pipinstall-e'.[test]'To run the tests:pytest
Package: llmopenai
🔥ChatLLM 基于知识库🔥Installpipinstall-UchatllmDocsUsagesfromchatllm.applicationsimportChatBaseqa=ChatBase()qa.load_llm(model_name_or_path="THUDM/chatglm-6b")foriinqa(query='周杰伦是谁',knowledge_base='周杰伦是傻子'):print(i,end='')# 根据已知信息无法回答该问题，因为周杰伦是中国内地流行歌手、演员、音乐制作人、导演，# 是具有一定的知名度和专业能力的人物，没有提供足够的信息无法判断他是傻子。OpenaiEcosystemClick to 无缝对接openai生态# 服务端pipinstall"chatllm[openai]"&&chatllm-runopenai<本地模型地址>SDK：pip install openaiimportopenaiopenai.api_base='http://0.0.0.0:8000/v1'openai.api_key='chatllm'prompt="你好"completion=openai.Completion.create(prompt=prompt,stream=True,model="text-davinci-003")forcincompletion:print(c.choices[0].text,end='')# 你好👋!我是人工智能助手 ChatGLM-6B,很高兴见到你，欢迎问我任何问题。客户端：点击下载chatbox，也可接入客户端openai_keys:不定期更新免费keysChatPDFClick to ChatPDFpipinstall"chatllm[pdf]"&&chatllm-runwebui--namechatpdfpython交互fromchatllm.applications.chatpdfimportChatPDFqa=ChatPDF(encode_model='nghuyong/ernie-3.0-nano-zh')qa.load_llm(model_name_or_path="THUDM/chatglm-6b")qa.create_index('财报.pdf')# 构建知识库foriinqa(query='东北证券主营业务'):print(i,end='')# 根据已知信息，东北证券的主营业务为证券业务。公司作为证券公司，主要从事证券经纪、证券投资咨询、与证券交易、# 证券投资活动有关的财务顾问、证券承销与保荐、证券自营、融资融券、证券投资基金代销和代销金融产品待业务。支持召回结果查看DeployClick to DeployChatGLM-6B 模型硬件需求|量化等级|最低 GPU 显存（推理） |最低 GPU 显存（高效参数微调） |
| -------------- | ------------------------- | --------------------------------- |
| FP16（无量化） | 13 GB                     | 14 GB                             |
| INT8           | 8 GB                     | 9 GB                             |
| INT4           | 6 GB                      | 7 GB                              |从本地加载模型安装指南ChatGLM-6B Mac 本地部署实操记录THUDM/ChatGLM-6B#从本地加载模型TODOClick to TODOChatLLM 应用接入非结构化文档（已支持 md、pdf、docx、txt 文件格式）搜索引擎与本地网页接入结构化数据接入（如 csv、Excel、SQL 等）知识图谱/图数据库接入增加 ANN 后端，ES/RedisSearch【确保生产高可用】增加多级缓存缓存多路召回问标量匹配多种向量化，向量匹配增加相似问，换几个问法高置信度直接返回答案【匹配标准问】答高置信度篇章增加上下文信息增加夸篇章信息增加召回信息的相似信息提前生成标准问，匹配问拒绝推断增加更多 LLM 模型支持THUDM/chatglm-6bTHUDM/chatglm-6b-int8THUDM/chatglm-6b-int4THUDM/chatglm-6b-int4-qeClueAI/ChatYuan-large-v2增加更多 Embedding 模型支持nghuyong/ernie-3.0-nano-zhnghuyong/ernie-3.0-base-zhshibing624/text2vec-base-chineseGanymedeNil/text2vec-large-chinese增加一键启动 webui利用 streamlit 实现 ChatPDF，一键启动chatllm-run webui --name chatpdf利用 gradio 实现 Web UI DEMO添加输出内容及错误提示引用标注增加知识库管理选择知识库开始问答上传文件/文件夹至知识库删除知识库中文件增加 API 支持利用 Fastapi/Flask/Grpc
实现流式接口chatllm-run openai <本地模型地址> --host 127.0.0.1 --port 8000前后端分离，实现调用 API 的 Web UI Demo交流群若二维码失效加微信拉群 313303303