Infosys Transformer Foundry solution provides buildings blocks for managing LLM Ops and model life cycle management such as model selection, finetuning, benchmarking, deployment at scale along with data pipelines.
- Model Zoo: Curated list of open source models along with their metadata, lifecycle status and model tagging.
- Leaderboard: LLM leaderboard for customer data (text, embedding and code) on public/private models for efficient selection of models.
- Benchmark tool: Allows benchmarking of fine-tuned or open source models.
- Data Pipelines: Allow users to create custom data processing workflows for their models.
- Fine Tuning: User can fine tune a model against custom datasets for tailored results.
- Model Deployment: Facilitate deployment of curated or fine-tuned models and create access points.
- Model Playground: Allows users to test the performance of different models available in the model zoo.
- RAG Playground: User can ingest documents in real time and leverage RAG for inferencing from them.
- Dataset Registration: Users have the ability to save datasets, they can then use them during finetuning or benchmarking jobs.
Find the hardware and software requirements here
a. Clone the GitHub repository
git clone -b < branch and Repo url>
b. Build docker container for each component
Navigate to each component folder within the repository and build the corresponding Docker image using the following command:
cd <component_folder>
docker build -t ${DOCKER_REPO}/<image_name>
c. Run the Docker Container
docker-compose -f docker-compose.yaml up
We appreciate your feedbacks, questions or bug reporting regarding this project. When posting issues in GitHub, ensure the posted examples follow the guidelines below:
Minimal: Provide the smallest possible code snippet that still reproduces the problem.
Complete: Include all necessary information (code, configuration, etc.) for someone else to replicate the issue.
Reproducible: Test your provided code to confirm it consistently reproduces the problem.