Chatting with GPT: Exploring Open-Source Large Language Models

Text Generation WebUI, stable vicuna, mpt-7b, opt, pythia, Text-to-image, stable diffusion

Large language models are fun to play with. It is even better to experiment with LLMs with web UI. I tried on Ubuntu 22.04.

Text generation webui

GitHub - oobabooga/text-generation-webui: A gradio web UI for running Large Language Models like…

Install

# install miniconda
curl -sL "https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh" > "Miniconda3.sh"
bash Miniconda3.sh
# exit shell and relogin
# create conda environment
conda create -n textgen python=3.10.9
conda activate textgen
# install pytorch
pip3 install pyyaml
pip3 install torch torchvision torchaudio
# install web ui
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt
# download model to text-generation-webui/models folder
# 26GB
python download-model.py TheBloke/stable-vicuna-13B-HF
# 13GB
python download-model.py mosaicml/mpt-7b-chat
# start webui
conda activate textgen
cd text-generation-webui
# use 0.0.0.0 if running on EC2, Azure VM, GCP Compute engine to make it reachable from Internet
python server.py --listen --listen-host 0.0.0.0 --listen-port 7860

Now access from http://<public ip>:7860/?__theme=dark

Stable Vicuna

https://www.mlexpert.io/prompt-engineering/stable-vicuna

TheBloke/stable-vicuna-13B-HF is one model that already combines original LLaMa with stable vicuna. (error: RuntimeError: probability tensor contains either `inf`, `nan` or element < 0)

Special prompt

### Human: <YOUR PROMPT>
### Assistant:

MPT-7B-CHAT

ValueError: Loading models/mosaicml_mpt-7b-chat requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.

Seems no UI option to set trust_remote_code=True. Although I can modify Python file, I decided to skip.

OPT 2.7B

python download-model.py
# then select b

Pythia-1.4B-deduped

stable diffusion web ui

GitHub - AUTOMATIC1111/stable-diffusion-webui: Stable Diffusion web UI

Install

sudo apt update -y
sudo apt install -y wget git python3 python3-venv libgl1
bash <(wget -qO- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)
# https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1742
# in webui-user.sh line 8:
# Commandline arguments for webui.py, for example: export COMMANDLINE_ARGS="--medvram --opt-split-attention"
export COMMANDLINE_ARGS="--skip-torch-cuda-test  --listen"
# https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/3082
# COMMAND_LINE= variable in webui-user.sh, put --precision full --no-half
# https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1349

You can access http://<public ip>:7860/

Appendix