Large language model serving and chatbot — Huggingface Chat UI overview Huggingface Chat UI allows you to deploy your own ChatGPT-like conversational UI that can interact with models on Huggingface, Huggingface text generation inference or custom API powered by LLM. For practitioners in LLM space, what are the options to stand up conversational UI which wire up to LLM? Streamlit or…

4bit quantization and Multi-GPU — TLDR TII’s Falcon 180B is released to HuggingFace (open access for research and commercial use, pretrained model performs better than Llama2–70b, so interesting to see the performance after fine-tuning). It takes 2 80GB A100 to load with 4bit quantization (more hardware requirements, here we use Azure Standard_NC48ads_A100_v4). Inference time is 10+…

Machine learning articles that are interesting to read — General Automated Feature Engineering in Python A guide to augmenting your dataset with new and informative features using Upginitowardsdatascience.com Upgini can search public and premium external data sources to enrich our features for LLM or neural network models. Can also use OpenAI GPT to generate features. LLM LLM Ecosystem When Should You Fine-Tune LLMs? There has been a flurry of exciting open-source LLMs which can be fine-tuned. But how does that compare to just using a…towardsdatascience.com

Machine learning articles that are interesting to read — MLOps How to Measure Drift in ML Embeddings We evaluated five embedding drift detection methodstowardsdatascience.com Data used in training might be different than data in real-world and can cause ML model performance degradation. It is important to detect this type of change to pre-take actions. What type of issues can you detect? For example, if your model is doing text classification, you might want to…

Yet another open-source large language model fine-tuning — If you are in Generative AI field like me, you must have heard LLaMA, Alpaca, Vicuna. They all derive from Facebook LLaMA model, enabling you to create your own “ChatGPT”. However, they all have one issue, the original LLaMA model is not released for commericial use, hence its descedants. …

Search with meaning, intent and context — This is the fifth article of building LLM-powered AI applications series. From the previous article, we introduced semantic search and applications. In this article, we will dive into technical details of semantic search. Technical mechanisms Concepts Vector similarity Vector Similarity Explained | Pinecone Vector embeddings have proven to be an effective tool in a variety of fields, including natural language processing and…www.pinecone.io

Search with meaning, intent and context — This is the fourth article of building LLM-powered AI applications series. From the previous article, we introduced embeddings/vectors which are the enabling technologies for semantic search. We introduced semantic search in second article of the series. A quick brief: semantic search understands the searcher’s intent and the contextual meaning of…

Machine learning articles that are interesting to read — General 30 Must-Subscribe Data and AI Newsletters in 2023 A curated list of reputable newsletters with quality content on DS, DE, ML and AI.towardsdatascience.com A list of data and AI newsletters to stay up-to-date Is F1-Score Really Better than Accuracy? What’s the cost of being wrong (and the gain of being right) according to different metricstowardsdatascience.com

Hidden secret to empower semantic search — This is the third article of building LLM-powered AI applications series. From the previous article, we know that in order to provide context to LLM, we need semantic search and complex query to find relevant context (traditional keyword search, full-text search won’t be enough). …