RAG to Riches: Revolutionizing Submittal Reviews with AI

This project aims to leverage Retrieval-Augmented Generation (RAG) to streamline the process of searching and reviewing engineering project manuals and contractor submittals for compliance with project requirements in the Architecture, Engineering and Construction (AEC) industry. 
 

Objectives


Objectives 
The final deliverable will be a web application and a command line interface (CLI) tool that can be used to query a project manual against a contractor submittal and report key information to the user. 

Milestones 
1. Document segmentation and query-based retrieval: 
  a. Project Manuals are large documents that easily exceed today’s Large Language Model (LLM) context windows, so they need to be broken into smaller, logical chunks. The tool will need to be able to read, store, and manipulate the data provided by the user to achieve this task. 
  b. Students will need to experiment with the size and organization of chunks to get the most reliable results. 
  c. By generating embeddings for users’ content and storing them into a vector database, the tool will be able to perform semantic search and retrieval on the data. 
  d. At the end of Milestone #1, a CLI proof of concept will allow a user to specify a target document on their machine, provide a natural language question, and the tool should respond with a section heading and page number where the answer is most likely to exist in the document. 


2. Large Language Model integration: 
a. With a project manual serving as Context, the tool will connect to an LLM via an API to provide meaningful, structured answers to the user’s queries.  
b. The API will aid in processing structured and unstructured data for generating answers. 
c. When a user asks a question, the tool will retrieve the most similar chunk (or chunks) of context to their query, and it will inject this data into a structured prompt for an LLM. 
d. Students will need to experiment with prompting the LLM to respond in a structure and tone that is meaningful to the users.  
e. At the end of Milestone #2, the CLI tool will allow the user to have a conversation with their specified document. The user will be able to ask questions about the document and the tool will respond with context-aware answers generated by the LLM. 

3. The LLM Analyzes the Submittal: 
a. The goal of this Milestone is to minimize the prompting necessary for a user and allow the LLM to perform its own analysis of a project manual against a contractor submittal. 
b. When a project manual and submittal is uploaded by the user, the tool should cross-reference the contents of the submittal against available chunks of the project manual. The most applicable chunks will be presented to the user as options to dive more deeply into the analysis of the submittal. 
c. Project Manuals are like rule books, each section of a project manual can be individually posed as a question or prompts to the LLM. Using the context provided by the user-selected chunks of the project, the LLM will generate a structured list of questions to ask of the submittal document. 
d. Using this list of questions, the LLM will prompt itself, iterating through the list of questions, and aggregate the responses. It will report these responses as results to the engineer with citations. 
e. At the end of Milestone #3, the CLI Tool will accept two files, a Project Manual and a Submittal, it will recommend Project Manual sections based on the submittal content, and based on the user’s choice of sections, it will aggregate prompts and responses into an output file. 


4. Develop a Web Application: 
a. The web app will have 2 interfaces. 
  i. File Q&A 
     1. Two document viewers side-by-side with file upload functions 
     2. Below each document viewer will be a chat interface for Q&A with its respective document. 
  ii. File Analysis 
     1. Two document viewers side-by-side with file upload functions. 
     2. Once both documents are uploaded, the interface offers the related sections from the Project Manual which match the Submittal. 
     3. The user should be able to select the related sections they are interested in using for analysis. 
     4. Selecting sections will scroll the project manual document to where the selection came from. 
     5. Once the user selects their chosen sections, a RUN button becomes available. 
     6. Run will trigger the functions developed in Milestone #3 and store it in a separate, toggleable RESULTS tab. 

Motivations


In the Architecture, Engineering, and Construction (AEC) industry, the submittal review process is a critical yet time-consuming task. When contractors purchase materials and products for a construction project, they submit detailed documents, known as submittals, to the project engineers for approval. These submittals include product data, shop drawings, and samples that must be meticulously reviewed to ensure they meet the project’s specifications and standards. 

Currently, this process is largely manual and involves sifting through extensive project manuals and technical documents. Engineers must verify that each submittal complies with the specified requirements, which can be a daunting and error-prone task given the volume and complexity of the information. Delays or mistakes in this process can lead to significant project setbacks, increased costs, and potential safety issues. An AI-driven tool can quickly search through vast amounts of 
documentation, accurately identify relevant information, and streamline the review process. This not only saves time and reduces the risk of human error but also ensures that projects stay on schedule and within budget. Solving this problem is crucial for enhancing the efficiency and reliability of construction projects. 

Qualifications


Minimum Qualifications:
  • Experience with Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs).
  • Experience with web development.

Preferred Qualifications:
  • Experience with the submittal review process from an applicable internship (we can train the process if necessary).


Details


Project Partner:

Brett Susany

NDA/IPA:

No Agreement Required

Number Groups:

2

Project Status:

Accepting Applicants

Website:
www.ssoe.com
Keywords:
WebArtificial Intelligence (AI)Consultancy
Card Image Capstone