DTA-SLO

DTA-SLO: Towards Efficient Multi-Chain Resource Management: A Dynamic Traffic-Aware Framework for SLO-Driven Microservice Optimization

A robust framework for optimizing resource management in complex microservice deployments with shared services and multi-chain dependencies. DTA-SLO leverages real-time traffic monitoring, time-series prediction using GRU (Gated Recurrent Unit), and novel optimization approaches to dynamically allocate resources based on individual service chain SLOs.

🌟 Key Features

• Real-time Traffic Monitoring: Continuous monitoring of service metrics including request rates, latency, and error rates
• ML-Powered Traffic Prediction: Advanced time-series forecasting using GRU neural networks
• Dynamic Resource Optimization: Intelligent resource allocation based on current and predicted workloads
• SLO-Driven Management: Ensures service level objectives are met across all service chains
• Kubernetes Integration: Native support for Kubernetes deployments with rolling updates
• Multi-Chain Awareness: Handles complex dependencies between microservices
• Performance Impact Analysis: Quantifies the impact of shared services on overall system performance

🚀 Quick Start

Prerequisites

• Python 3.8+
• Kubernetes cluster (for deployment)
• Docker (optional)
• kubectl configured with cluster access

Installation

1. Clone the repository:

   git clone https://github.com/ahmadpanah/dta-slo.git
   cd dta-slo
   

2. Create and activate virtual environment:

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   

3. Install dependencies:

   pip install -r requirements.txt
   

🐳 Docker Deployment

Using Docker Compose

docker-compose up --build

Or using Docker directly

docker build -t dta-slo .
docker run -d --name dta-slo -v ~/.kube:/home/appuser/.kube:ro -p 8000:8000 dta-slo

Basic Usage

1. Configure your services in config/config.yaml: ```yaml services: auth-service: chains:
   • user-registration
   • order-processing min_cpu: 0.1 max_cpu: 4.0 min_memory: 128 max_memory: 8192

slos: user-registration: 0.2 # 200ms order-processing: 0.5 # 500ms

2. Run the framework:
```bash
python main.py

📁 Project Structure

dta_slo/
│
├── __init__.py
├── requirements.txt
├── config/
│   └── config.yaml
│
├── src/
│   ├── __init__.py
│   ├── models/
│   │   ├── data_models.py
│   │   └── metrics.py
│   │
│   ├── components/
│   │   ├── traffic_monitor.py
│   │   ├── traffic_predictor.py
│   │   ├── performance_quantifier.py
│   │   └── resource_allocator.py
│   │
│   └── utils/
│       └── kubernetes_utils.py
│
└── main.py

🔧 Core Components

Traffic Monitor

Collects and processes real-time service metrics:

traffic_monitor = TrafficMonitor(sampling_interval=1.0)
traffic_monitor.record_request("auth-service", "user-registration", 0.15)

Traffic Predictor

Forecasts future traffic patterns using GRU:

predictor = TrafficPredictor(sequence_length=60, prediction_horizon=10)
future_load = predictor.predict(historical_data)

Resource Allocator

Optimizes resource allocation based on traffic and SLOs:

allocator = DynamicResourceAllocator(traffic_monitor, traffic_predictor, performance_quantifier)
new_allocations = allocator.optimize_resources(services, slos)

📊 Performance Metrics

DTA-SLO tracks several key metrics:

• Resource Utilization
• SLO Violation Rate
• Average End-to-End Latency
• Fairness Index (resource distribution)

🔍 Configuration Options

Monitoring Settings

monitoring:
  sampling_interval: 1.0  # seconds
  window_size: 3600      # seconds

Prediction Settings

prediction:
  sequence_length: 60
  prediction_horizon: 10
  feature_dim: 3

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📧 Contact

• Seyed Hossein Ahmadpanah - h.ahmadpanah@iau.ac.ir
• Meghdad Mirabi - meghdad.mirabi@cs.tu-darmstadt.de

This site is open source. Improve this page.