pw.xpacks.llm.question_answering

Builds the logic and the API for adaptive RAG application.

It allows to build a RAG app with Pathway vector store and Pathway components. Gives the freedom to choose between two question answering strategies, short (concise), and long (detailed) response, that can be set during the post request. Allows for LLM agnosticity with freedom to choose from proprietary or open-source LLMs.

It differs from BaseRAGQuestionAnswerer in adaptive choosing the number of chunks used as a context of a question. First, only n_starting_documents chunks are used, and then the number is increased until an answer is found.

• Parameters
  • llm (BaseChat) – LLM instance for question answering. See https://pathway.com/developers/api-docs/pathway-xpacks-llm/llms for available models.
  • indexer (VectorStoreServer | DocumentStore) – Indexing object for search & retrieval to be used for context augmentation.
  • default_llm_name (str | None) – Default LLM model to be used in queries, only used if model parameter in post request is not specified. Omitting or setting this to None will default to the model name set during LLM’s initialization.
  • short_prompt_template (UDF) – Template for document question answering with short response. A pw.udf function is expected. Defaults to pathway.xpacks.llm.prompts.prompt_short_qa.
  • long_prompt_template (UDF) – Template for document question answering with long response. A pw.udf function is expected. Defaults to pathway.xpacks.llm.prompts.prompt_qa.
  • summarize_template (UDF) – Template for text summarization. Defaults to pathway.xpacks.llm.prompts.prompt_summarize.
  • n_starting_documents (int) – Number of documents embedded in the first query.
  • factor (int) – Factor by which a number of documents increases in each next query, if an answer is not found.
  • max_iterations (int) – Number of times to ask a question, with the increasing number of documents.
  • strict_prompt (bool) – If LLM should be instructed strictly to return json. Increases performance in small open source models, not needed in OpenAI GPT models.

Example:

import pathway as pw  
from pathway.xpacks.llm import embedders, splitters, llms, parsers  
from pathway.xpacks.llm.vector_store import VectorStoreServer  
from pathway.udfs import DiskCache, ExponentialBackoffRetryStrategy  
from pathway.xpacks.llm.question_answering import AdaptiveRAGQuestionAnswerer  
my_folder = pw.io.fs.read(
    path="/PATH/TO/MY/DATA/*",  # replace with your folder
    format="binary",
    with_metadata=True)  
sources = [my_folder]  
app_host = "0.0.0.0"  
app_port = 8000  
parser = parsers.ParseUnstructured()  
text_splitter = splitters.TokenCountSplitter(max_tokens=400)  
embedder = embedders.OpenAIEmbedder(cache_strategy=DiskCache())  
vector_server = VectorStoreServer(  
    *sources,
    embedder=embedder,
    splitter=text_splitter,
    parser=parser,
)
chat = llms.OpenAIChat(  
    model=DEFAULT_GPT_MODEL,
    retry_strategy=ExponentialBackoffRetryStrategy(max_retries=6),
    cache_strategy=DiskCache(),
    temperature=0.05,
)
app = AdaptiveRAGQuestionAnswerer(  
    llm=chat,
    indexer=vector_server,
)
app.build_server(host=app_host, port=app_port)  
app.run_server()

answer_query(pw_ai_queries)

sourceCreate RAG response with adaptive retrieval.

build_server(host, port, **rest_kwargs)

sourceAdds HTTP connectors to input tables, connects them with table transformers.

serve_callable(route, schema=None, retry_strategy=None, cache_strategy=None, **additional_endpoint_kwargs)

sourceServe additional endpoints by wrapping callables. Expects an endpoint route. Schema is optional, adding schema type will enforce the

webserver to check arguments.

Beware that if Schema is not set, incorrect types may cause runtime error.

Example:

@rag_app.serve_callable(route="/agent")  
async def some_func(user_query: str) -> str:
    # define your agent, or custom RAG using any framework or plain Python
    # ...
    messages = [{"role": "user", "content": user_query}]
    result = agent.invoke(messages)
    return result

summarize_query(summarize_queries)

sourceFunction for summarizing given texts.

Builds the logic and the API for basic RAG application.

Base class to build RAG app with Pathway vector store and Pathway components. Gives the freedom to choose between two question answering strategies, short (concise), and long (detailed) response, that can be set during the post request. Allows for LLM agnosticity with freedom to choose from proprietary or open-source LLMs.

• Parameters
  • llm (BaseChat) – LLM instance for question answering. See https://pathway.com/developers/api-docs/pathway-xpacks-llm/llms for available models.
  • indexer (VectorStoreServer | DocumentStore) – Indexing object for search & retrieval to be used for context augmentation.
  • default_llm_name (str | None) – Default LLM model to be used in queries, only used if model parameter in post request is not specified. Omitting or setting this to None will default to the model name set during LLM’s initialization.
  • short_prompt_template (UDF) – Template for document question answering with short response. A pw.udf function is expected. Defaults to pathway.xpacks.llm.prompts.prompt_short_qa.
  • long_prompt_template (UDF) – Template for document question answering with long response. A pw.udf function is expected. Defaults to pathway.xpacks.llm.prompts.prompt_qa.
  • summarize_template (UDF) – Template for text summarization. Defaults to pathway.xpacks.llm.prompts.prompt_summarize.
  • search_topk (int) – Top k parameter for the retrieval. Adjusts number of chunks in the context.

Example:

import pathway as pw  
from pathway.xpacks.llm import embedders, splitters, llms, parsers  
from pathway.xpacks.llm.vector_store import VectorStoreServer  
from pathway.udfs import DiskCache, ExponentialBackoffRetryStrategy  
from pathway.xpacks.llm.question_answering import BaseRAGQuestionAnswerer  
from pathway.xpacks.llm.servers import QASummaryRestServer 
my_folder = pw.io.fs.read(
    path="/PATH/TO/MY/DATA/*",  # replace with your folder
    format="binary",
    with_metadata=True)  
sources = [my_folder]  
app_host = "0.0.0.0"  
app_port = 8000  
parser = parsers.ParseUnstructured()  
text_splitter = splitters.TokenCountSplitter(max_tokens=400)  
embedder = embedders.OpenAIEmbedder(cache_strategy=DiskCache())  
vector_server = VectorStoreServer(  
    *sources,
    embedder=embedder,
    splitter=text_splitter,
    parser=parser,
)
chat = llms.OpenAIChat(  
    model=DEFAULT_GPT_MODEL,
    retry_strategy=ExponentialBackoffRetryStrategy(max_retries=6),
    cache_strategy=DiskCache(),
    temperature=0.05,
)
rag = BaseRAGQuestionAnswerer(  
    llm=chat,
    indexer=vector_server,
)
app = QASummaryRestServer(app_host, app_port, rag)  
app.run_server()

answer_query(pw_ai_queries)

sourceMain function for RAG applications that answer questions based on available information.

build_server(host, port, **rest_kwargs)

sourceAdds HTTP connectors to input tables, connects them with table transformers.

serve_callable(route, schema=None, retry_strategy=None, cache_strategy=None, **additional_endpoint_kwargs)

sourceServe additional endpoints by wrapping callables. Expects an endpoint route. Schema is optional, adding schema type will enforce the

webserver to check arguments.

Beware that if Schema is not set, incorrect types may cause runtime error.

Example:

@rag_app.serve_callable(route="/agent")  
async def some_func(user_query: str) -> str:
    # define your agent, or custom RAG using any framework or plain Python
    # ...
    messages = [{"role": "user", "content": user_query}]
    result = agent.invoke(messages)
    return result

summarize_query(summarize_queries)

sourceFunction for summarizing given texts.

Class for slides search.

answer_query(pw_ai_queries)

sourceReturn similar docs from the index.

build_server(host, port, **rest_kwargs)

sourceAdds HTTP connectors to input tables, connects them with table transformers.

serve_callable(route, schema=None, retry_strategy=None, cache_strategy=None, **additional_endpoint_kwargs)

sourceServe additional endpoints by wrapping callables. Expects an endpoint route. Schema is optional, adding schema type will enforce the

webserver to check arguments.

Beware that if Schema is not set, incorrect types may cause runtime error.

Example:

@rag_app.serve_callable(route="/agent")  
async def some_func(user_query: str) -> str:
    # define your agent, or custom RAG using any framework or plain Python
    # ...
    messages = [{"role": "user", "content": user_query}]
    result = agent.invoke(messages)
    return result

summarize_query(summarize_queries)

sourceFunction for summarizing given texts.

Connector for interacting with the Pathway RAG applications. Either (host and port) or url must be set.

• Parameters
  • host (-) – The host of the RAG service.
  • port (-) – The port of the RAG service.
  • url (-) – The URL of the RAG service.
  • timeout (-) – Timeout for requests in seconds. Defaults to 90.
  • additional_headers (-) – Additional headers for the requests.

pw_ai_answer(prompt, filters=None, model=None)

sourceReturn RAG answer based on a given prompt and optional filter.

• Parameters
  • prompt (-) – Question to be asked.
  • filters (-) – Optional metadata filter for the documents. Defaults to None, which means there will be no filter.
  • model (-) – Optional LLM model. If None, app default will be used by the server.

pw_ai_summary(text_list, model=None)

sourceSummarize a list of texts.

• Parameters
  • text_list (-) – List of texts to summarize.
  • model (-) – Optional LLM model. If None, app default will be used by the server.

pw_list_documents(filters=None, keys=['path'])

sourceList indexed documents from the vector store with optional filtering.

• Parameters
  • filters (-) – Optional metadata filter for the documents.
  • keys (-) – List of metadata keys to be included in the response. Defaults to \["path"\]. Setting to None will retrieve all available metadata.

retrieve(query, k=3, metadata_filter=None, filepath_globpattern=None)

sourceRetrieve closest documents from the vector store based on a query.

• Parameters
  • query (-) – The query string.
  • k (-) – The number of results to retrieve.
  • metadata_filter (-) – Optional metadata filter for the documents. Defaults to None, which means there will be no filter.
  • filepath_globpattern (-) – Glob pattern for file paths.

statistics()

sourceRetrieve stats from the vector store.