last updated: 3/1/2024 11:00pm

Advanced RAG for Data Caching

Overview

We are developing an innovative database aimed at revolutionizing the way Long Language Models (LLMs) interact with data visualization and analysis. Our project, Advanced RAG for Data Caching, strives to provide an ultimate collection of LLM prompt templates specifically designed for Python plotting graphs for datasets.

The Problem

Current code interpreters, such as PandasAI, are proficient at generating data graphs for datasets based on simple instructions (e.g., scatter plots, bar plots, linear regression). However, they falter when tasked with more complex requests, such as constructing a K-Nearest Neighbors (KNN) machine learning model for a specific dataset provided by the user.

Our Solution

Our team possesses extensive knowledge in structuring prompt templates to guide LLMs in code generation, alongside a deep understanding of various ML and statistical models for advanced purposes. Leveraging this expertise, we aim to develop an Advanced Retrieval-Augmented Generation (RAG) system equipped with an extensive repository of template models and instructions for complex analytical tasks. This system, which we term Data Caching, is designed to facilitate easy retrieval of sophisticated analytical methods by users.

Impact

The implementation of this system will mark a significant milestone in data analysis and LLM interaction. It will enable users to dynamically contribute new, advanced, and improved templates for LLMs, enhancing the capability to generate complex analytical solutions. Users will be able to efficiently search for specific methods or models using concise descriptions, populate the prompt template with their dataset, and receive clear, editable code templates tailored to their analysis needs. This streamlined process promises to deliver optimal results swiftly and effectively, transforming the landscape of data analysis and machine learning model development.