Generative AI/Large Language Model Focus Areas Part 1

List of interested topics with Learning Resources

Part 2 is here.

Application building

Emerging Large Language Model (LLM) Application Architecture

Design Principles for Generative AI Applications

Toward General Design Principles for Generative AI Applications

Content-Centric Prototyping of Generative AI Applications: Emerging Approaches and Challenges in…

Prompt engineering techniques

General

Prompt Engineering a Prompt Engineer

Exploring Prompt Engineering Practices in the Enterprise

Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review

A Brief History of Prompt: Leveraging Language Models. (Through Advanced Prompting)

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language…

Large Language Models Are Human-Level Prompt Engineers

Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering

Prompt Design and Engineering: Introduction and Advanced Methods

A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications

A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges

An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's…

In-context learning

In-Context Learning in Large Language Models: A Neuroscience-inspired Analysis of Representations

A Survey on In-context Learning

reasoning

Towards Reasoning in Large Language Models: A Survey

A Survey of Reasoning with Foundation Models

Strategic Reasoning with Language Models

Reasoning with Large Language Models, a Survey

Logical Reasoning in Large Language Models: A Survey

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Teaching Large Language Models to Self-Debug

SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning

Self-Evaluation Improves Selective Generation in Large Language Models

Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models

chain of thought

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future

Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective

graph of thought

Graph of Thoughts: Solving Elaborate Problems with Large Language Models

tree of thought

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

self consistency

Self-Consistency Improves Chain of Thought Reasoning in Language Models

step back

Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models

ReAct

ReAct Prompting | Prompt Engineering Guide

PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents

ReAct: Synergizing Reasoning and Acting in Language Models

ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent

MetaPrompting

MetaPrompting: Learning to Learn Better Prompts

Towards Goal-oriented Large Language Model Prompting: A Survey

Meta Prompting for AGI Systems

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding

On Meta-Prompting

Misc

Guide to Anthropic's prompt engineering resources

Prompt engineering techniques with Azure OpenAI - Azure OpenAI Service

12 Prompt Engineering Techniques

Self-Refine: Iterative Refinement with Self-Feedback

Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models

Prompting framework

Prompting Frameworks for Large Language Models: A Survey

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4

Integration with Knowledge graph

Large Language Models on Graphs: A Comprehensive Survey

Unifying Large Language Models and Knowledge Graphs: A Roadmap

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

Graph Retrieval-Augmented Generation: A Survey

Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for…

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications

A Survey of Large Language Models for Graphs

Integration with Recommender System

How Can Recommender Systems Benefit from Large Language Models: A Survey

A Survey on Large Language Models for Recommendation

Large Language Models for Generative Recommendation: A Survey and Visionary Discussions

Recommender Systems in the Era of Large Language Models (LLMs)

A Survey on Large Language Models for Personalized and Explainable Recommendations

A Review of LLM-based Explanations in Recommender Systems

Retrieval

Retrieval-Augmented Generation for Large Language Models: A Survey

Retrieval-Augmented Generation for AI-Generated Content: A Survey

A Survey on Retrieval-Augmented Text Generation

A Survey on Retrieval-Augmented Text Generation for Large Language Models

A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and…

A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models

A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

Active Retrieval Augmented Generation

The Chronicles of RAG: The Retriever, the Chunk and the Generator

Seven Failure Points When Engineering a Retrieval Augmented Generation System

Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs…

Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey

Learning to Retrieve In-Context Examples for Large Language Models

Related Work and Citation Text Generation: A Survey

A Cheat Sheet and Some Recipes For Building Advanced RAG

The Ultimate Guide on Retrieval Strategies - RAG (part-4) - ChatGen

A Guide on 12 Tuning Strategies for Production-Ready RAG Applications

Building and Evaluating Advanced RAG Applications

Embedding

Improving Text Embeddings with Large Language Models

Embedding in Recommender Systems: A Survey

Embedding in Recommender Systems: A Survey

Vector database

A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge

Survey of Vector Database Management Systems

unstructured and unstructured entity extraction

Structured information extraction from complex scientific text with fine-tuned large language…

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

Large Language Models for Information Retrieval: A Survey

Orchestration framework

langchain

Introduction | 🦜️🔗 Langchain

llamaindex

Starter Tutorial - LlamaIndex 🦙 0.9.26

prompt flow

What is Azure Machine Learning prompt flow - Azure Machine Learning

semantic kernel

Create AI agents with Semantic Kernel

LLM agents

A Survey on Large Language Model based Autonomous Agents

A Survey on LLM-Based Agents: Common Workflows and Reusable LLM-Profiled Components

A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and…

Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects

The Rise and Potential of Large Language Model Based Agents: A Survey

Understanding the planning of LLM agents: A survey

A Survey on Self-Evolution of Large Language Models

The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A…

Tool Learning with Large Language Models: A Survey

Reflexion: Language Agents with Verbal Reinforcement Learning

LLM Multi-Agent Systems: Challenges and Open Problems

Large Language Model based Multi-Agents: A Survey of Progress and Challenges

function calling

https://platform.openai.com/docs/guides/function-calling

An LLM Compiler for Parallel Function Calling

tools

Tools | 🦜️🔗 Langchain

Building Custom Tools for LLM Agents | Pinecone

autogen, autogpt

From AutoGPT to AGI: The Evolutionary Journey of AutoGen

Conversational AI chatbot (conversational memory)

Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM…

MemGPT: Towards LLMs as Operating Systems

Augmenting Language Models with Long-Term Memory

RET-LLM: Towards a General Read-Write Memory for Large Language Models

MemoryBank: Enhancing Large Language Models with Long-Term Memory

Conversational Memory for LLMs with Langchain | Pinecone

Task
LLM for software development

Large Language Models for Software Engineering: A Systematic Literature Review

Microsoft Copilot app/plugins

Overview of Plugins for Microsoft Copilot

ConfusedPilot: Confused Deputy Risks in RAG-based LLMs

https://www.microsoft.com/en-us/research/uploads/prod/2023/12/AI-and-Productivity-Report-First-Edition.pdf

https://www.microsoft.com/insidetrack/blog/uploads/prod/2024/04/Ch-2-story-1-Deploying-Copilot-for-Microsoft-365-internally-at-Microsoft.pdf

Large Language Models/LLMs

Understanding LLMs: A Comprehensive Overview from Training to Inference

Large Language Models: A Survey

AI-Generated Content (AIGC): A Survey

A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?

A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to…

Eight Things to Know about Large Language Models

From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial…

A Survey of Large Language Models

Large Language Models Meet NLP: A Survey

ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?

A Comprehensive Overview of Large Language Models

Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms…

Large Language Models for Generative Information Extraction: A Survey

Knowledge Mechanisms in Large Language Models: A Survey and Perspective

Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges

LLMOps

The Costly Dilemma: Generalization, Evaluation and Cost-Optimal Deployment of Large Language Models

A Survey on LLM-as-a-Judge

An Introduction to LLMOps: Operationalizing and Managing Large Language Models using Azure ML

LLM model evaluation

Evaluating Large Language Models: A Comprehensive Survey

A Survey on Evaluation of Large Language Models

A Survey of Useful LLM Evaluation

A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges…

LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods

Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering

List of Evals

RAG evaluation

RAG Evaluation

Evaluation of Retrieval-Augmented Generation: A Survey

RAGAS: Automated Evaluation of Retrieval Augmented Generation

Benchmarking Large Language Models in Retrieval-Augmented Generation

ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems

Benchmarking Large Language Models in Retrieval-Augmented Generation

Retrieval-Augmented Generation for Natural Language Processing: A Survey

An Overview on RAG Evaluation | Weaviate - Vector Database

Best Practices for LLM Evaluation of RAG Applications

LLM logging/monitoring

LLM Monitoring and Observability

How To Use LangChain With Monitoring To Fine-Tune Your LLM Applications

LLM Observability and Monitoring

LLM Monitoring: The Beginner's Guide | Lakera - Protecting AI teams that disrupt the world.

LangKit: Making Large Language Models Safe and Responsible

Open-source LLM

ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?

LLM360: Towards Fully Transparent Open-Source LLMs

A Comparison of DeepSeek and Other LLMs

The Open Source Advantage in Large Language Models (LLMs)

Integration with various cloud provider

Azure ml

Use Azure OpenAI models in Azure Machine Learning - Azure Machine Learning

Model Catalog and Collections - Azure Machine Learning

AWS bedrock

Build Generative AI Applications with Foundation Models - Amazon Bedrock - AWS

GCP vertex ai

Model information | Vertex AI | Google Cloud

Integration with container environment

docker

GitHub - docker/genai-stack: Langchain + Docker + Neo4j + Ollama

k8s

GitHub - cncf/llm-starter-pack: 🤖 Get started with LLMs on your kind cluster, today!

Deploy LLM models on Kubernetes using OpenLLM | Cloudraft

Serve an LLM with multiple GPUs in GKE | Kubernetes Engine | Google Cloud

openshift

LLMs, Chatbots, Talk with your Doc,... - AI on OpenShift

Fine-tuning and Serving an open source foundation model with Red Hat OpenShift AI

Appendix

Generative AI intro

https://www.freecodecamp.org/news/learn-prompt-engineering-full-course/
https://www.cloudskillsboost.google/course_templates/536

Microsoft Azure AI Fundamentals: Generative AI - Training

New technical deep dive course: Generative AI Foundations on AWS | Amazon Web Services

Large language model

Google Cloud Skills Boost

Google Cloud Skills Boost

Appendix

A Comprehensive Survey of Large Language Models (LLMs)

The Evolving LLM Landscape: 8 Key Trends to Watch