Deliverable: Project Initiation Document

The deliverable is a Project Initiation Document (PID), a formal report submitted to the instruction team and the company stakeholders. This document will serve as a foundational blueprint for your project and must include the following three components:

1. 
   Framework Deployment Confirmation: A screenshot demonstrating the successful deployment and operational status of the provided backend and frontend starter framework.


2. 
   LLM Strategy Plan: A comprehensive plan for the selection of an appropriate open-source Large Language Model (LLM). This plan should justify your choice based on project needs (e.g., performance, licensing, size).


3. 
   Key Term Alignment Design: A detailed design proposal outlining the methodology for aligning the company's specified key business terms with the structured data extracted from the relevant forms.

Purpose: Foundational Alignment and Technical Readiness

This milestone is critical to establishing a clear and correct starting point for the project. It serves two primary purposes:

1. 
   Strategic Alignment: To ensure that students immediately develop a solid and accurate understanding of the company’s specific business needs and requirements for term alignment.


2. 
   Technical Foundation: To confirm the technical readiness of the student by verifying their ability to successfully deploy and utilize the starter framework developed by a previous high-performing master's student.

Tasks: LLM Selection and Framework Integration

The following tasks must be completed by each individual team member:

1. 
   Open-Source LLM Selection: Select two distinct open-source Large Language Models (LLMs) from the provided list. The selection must be based on suitability for the member's designated computing environment (personal machine or university cluster).In teams of two or more members, individual LLM selections may overlap. However, the team must collectively select a minimum of one more LLM than the number of team members ($\text{K}+1$ unique LLMs for $\text{K}$ members).


2. 
   Framework Adaptation: Modify the existing starter framework to replace the placeholder commercial LLM API calls with local, on-device inference using the two selected open-source LLMs.


3. 
   UI Verification: Successfully execute and demonstrate the functionality of the revised web User Interface (UI) for each integrated open-source LLM separately.


4. 
   Data Processing and Extraction: Individually upload a designated sample data form through the revised web UI for each of the two integrated LLMs, ensuring the successful extraction of structured data.


5. 
   Output Capture: Save the resulting structured data output from each individual execution as a separate JSON file.

Deliverable: Source Code and Comparative Extraction Results

The formal deliverable includes both the technical assets and the resulting structured data:

1. 
   Revised Source Code: The complete, functional source code for the starter framework, modified to successfully run inference with the two selected open-source LLMs.


2. 
   Comparative JSON Results: A total of six (6) structured JSON files per team member capturing the output of the data extraction process. (Assuming one sample form is provided, the total is two (2) JSON files per member—one for each selected LLM).

Purpose: Core Skill Development and Functional Validation

The primary goal of this milestone is two-fold:

1. 
   Technical Proficiency: To develop practical, hands-on experience in local Large Language Model (LLM) deployment and the process of integrating open-source models into an existing application framework.


2. 
   Functional Validation: To functionally validate the revised starter framework's ability to perform structured data extraction using different open-source models, thereby providing a preliminary comparative analysis of their respective performance on a common task.

Tasks: Iterative Refinement and Benchmarking

The core activities for this sprint involve assessing initial performance, optimizing the model strategy, and rigorously documenting all experimental data:

1. 
   Digitalization Result Assessment: Conduct a thorough evaluation and qualitative analysis of the structured data output (the initial digitalization results) generated in the previous sprint.


2. 
   LLM Strategy Refinement: Based on the assessment, refine the open-source LLM selection or tuning strategy. This may involve exploring new models, optimizing existing model hyperparameters, or improving prompt engineering techniques.


3. 
   Experimentation and Documentation: Rigorously record all experiments and results for every LLM used by the individual or the team. This documentation must include the specific model version, any prompt templates used, and the raw extraction output (JSON files) or the calculated evaluation metrics.

Deliverable: Optimized System and Comprehensive Evaluation Report

The formal deliverables will demonstrate both the functional improvement of the system and the analytical rigor of the team:

1. 
   Refined Source Code: The optimized source code of the starter framework, incorporating all necessary changes for the refined LLM strategy (e.g., new models, prompt engineering, or post-processing logic).


2. 
   Comprehensive Evaluation Report: A detailed report containing the quantitative evaluation results. This report must include the calculated performance metrics (which will be specified by the instruction team) and clearly demonstrate the method used to align the extracted data schema tree with the company's designated key business terms.

Purpose: Performance Enhancement and Advanced Skill Acquisition

The primary goal of this milestone is to elevate technical proficiency beyond basic integration:

To improve AI system development skills by moving into the optimization phase. This involves exploring advanced techniques such as prompt engineering optimization, comparative benchmarking across diverse open-source LLMs, and formalizing the data alignment process to meet functional performance requirements.

Deliver final report, code, and presentation to company.

Reflect on outcomes, lessons learned, and possible extensions.