Pathway has been selected for Intel Liftoff

Intel Liftoff is a program designed to assist tech startups that are focusing on innovation in areas like artificial intelligence (AI), analytics, high-performance computing (HPC), and graphics.

Pathway is a scalable framework designed to build real-time intelligent applications, data pipelines, and integrate AI/ML models. Think document search for AI apps, live data pipelines for LLM analytics, AI-enabled vehicle telemetry tracking, augmented reality, and real-time monitoring. More precisely, Pathway is a high-throughput, low-latency data processing framework, powering Event Stream Processing and AI pipelines over data streams. It provides a Python-first developer experience and is powered by a scalable Rust engine. It deploys through containers, usually on Kubernetes.

Pathway is designed to handle modern AI use cases, focusing on real-time data integration and indexing for LLM RAG applications. Pathway promotes building systems in which the retriever and LLM are aware of the joint use case and jointly tuned: the retriever uses multiple indexes with feedback from the LLM for optimal context generation, while the LLM is tuned for optimal performance with a rich context.

Pathway is currently available in a self-service manner, at github.com/pathwaycom and as a Python package, now also rolling out on Docker Hub.

In an apples-to-apples comparison, Pathway is designed to take on Flink and beat it in terms of speed, developer experience, ease of deployment, and especially, suitability for AI use cases. This includes ML/AI-first capabilities such as feature serving, API support with query/answer capability, unstructured data ETL, powerful data indexing, and acting as a RAG backbone.

Intel Liftoff startups are blazing new trails in a wide range of AI applications and we are proud to be joining this vibrant community!