Añadir Home
commit
4ea81bd80f
|
|
@ -0,0 +1,282 @@
|
|||
# Base Project Library Documentation
|
||||
|
||||
## Overview
|
||||
A modular Python library designed for building applications that work with LLMs, translation services, and file processing, with a focus on Windows environments.
|
||||
|
||||
## Table of Contents
|
||||
1. [Installation](#installation)
|
||||
2. [Project Structure](#project-structure)
|
||||
3. [Core Components](#core-components)
|
||||
4. [Services](#services)
|
||||
5. [Utilities](#utilities)
|
||||
6. [Examples](#examples)
|
||||
|
||||
## Installation
|
||||
|
||||
### Requirements
|
||||
```
|
||||
Python 3.8+
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Required packages:
|
||||
```text
|
||||
python-dotenv>=1.0.0
|
||||
openai>=1.0.0
|
||||
google-cloud-translate>=3.0.0
|
||||
pandas>=2.0.0
|
||||
openpyxl>=3.0.0
|
||||
langid>=1.1.6
|
||||
tk>=0.1.0
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
```
|
||||
base_project/
|
||||
├── config/
|
||||
│ ├── __init__.py
|
||||
│ ├── api_keys.py # API key management
|
||||
│ ├── api_setup.py # GUI for API configuration
|
||||
│ └── settings.py # General settings
|
||||
│
|
||||
├── services/
|
||||
│ ├── llm/ # LLM Services
|
||||
│ │ ├── base.py
|
||||
│ │ ├── openai_service.py
|
||||
│ │ ├── grok_service.py
|
||||
│ │ ├── ollama_service.py
|
||||
│ │ ├── batch_processor.py
|
||||
│ │ └── llm_factory.py
|
||||
│ │
|
||||
│ ├── excel/ # Excel Services
|
||||
│ │ └── excel_service.py
|
||||
│ │
|
||||
│ └── translation/ # Translation Services
|
||||
│ ├── base.py
|
||||
│ ├── google_translate.py
|
||||
│ └── translation_factory.py
|
||||
│
|
||||
├── utils/
|
||||
│ ├── file_utils.py # File handling utilities
|
||||
│ ├── gui_utils.py # GUI utilities
|
||||
│ ├── logger_utils.py # Logging utilities
|
||||
│ └── progress_bar.py # Progress tracking
|
||||
│
|
||||
├── menu.py # Main application interface
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. API Key Management
|
||||
The APIKeyManager provides secure storage and retrieval of API keys:
|
||||
|
||||
```python
|
||||
from config.api_keys import APIKeyManager
|
||||
|
||||
# Get API keys
|
||||
openai_key = APIKeyManager.get_openai_key()
|
||||
google_key = APIKeyManager.get_google_key()
|
||||
|
||||
# Store new key
|
||||
APIKeyManager.store_key("openai", "your-key-here")
|
||||
```
|
||||
|
||||
### 2. GUI Interface
|
||||
The main menu provides a user-friendly interface:
|
||||
|
||||
```python
|
||||
from menu import MainMenu
|
||||
import tkinter as tk
|
||||
|
||||
def main():
|
||||
root = tk.Tk()
|
||||
app = MainMenu(root)
|
||||
root.mainloop()
|
||||
```
|
||||
|
||||
## Services
|
||||
|
||||
### LLM Services
|
||||
Modular services for different LLM providers with a unified interface.
|
||||
|
||||
#### Factory Pattern Usage
|
||||
```python
|
||||
from services.llm.llm_factory import LLMFactory
|
||||
|
||||
# Create service instance
|
||||
llm_service = LLMFactory.create_service("openai", model="gpt-4")
|
||||
|
||||
# Generate text
|
||||
response = llm_service.generate_text("Tell me a joke")
|
||||
```
|
||||
|
||||
#### Batch Processing
|
||||
Efficient processing of multiple items:
|
||||
|
||||
```python
|
||||
from services.llm.batch_processor import BatchProcessor, BatchConfig
|
||||
|
||||
# Configure batch processing
|
||||
config = BatchConfig(
|
||||
batch_size=20,
|
||||
progress_callback=lambda msg: print(msg)
|
||||
)
|
||||
|
||||
# Create processor
|
||||
processor = BatchProcessor(llm_service, config)
|
||||
|
||||
# Process items
|
||||
items = [{"text": "Process this"}, {"text": "And this"}]
|
||||
results = processor.process_batch(
|
||||
items=items,
|
||||
system_prompt="You are a helpful assistant.",
|
||||
template="Process these items: {items}"
|
||||
)
|
||||
```
|
||||
|
||||
### Translation Services
|
||||
Translation capabilities with multiple provider support.
|
||||
|
||||
#### Google Translate
|
||||
```python
|
||||
from services.translation.translation_factory import TranslationFactory
|
||||
|
||||
# Create translator
|
||||
translator = TranslationFactory.create_service("google")
|
||||
|
||||
# Translate text
|
||||
translated = translator.translate_text(
|
||||
"Hello world",
|
||||
target_language="es"
|
||||
)
|
||||
|
||||
# Batch translation
|
||||
texts = ["Hello", "World"]
|
||||
translations = translator.translate_batch(
|
||||
texts,
|
||||
target_language="es"
|
||||
)
|
||||
```
|
||||
|
||||
### Excel Services
|
||||
Robust Excel file handling with retry mechanism and formatting.
|
||||
|
||||
```python
|
||||
from services.excel.excel_service import ExcelService
|
||||
|
||||
# Create service
|
||||
excel_service = ExcelService()
|
||||
|
||||
# Read with retries
|
||||
df = excel_service.read_excel("input.xlsx")
|
||||
|
||||
# Save with formatting
|
||||
format_options = {
|
||||
'freeze_row': 2,
|
||||
'max_column_width': 50,
|
||||
'wrap_threshold': 50,
|
||||
}
|
||||
|
||||
excel_service.save_excel(
|
||||
df,
|
||||
"output.xlsx",
|
||||
format_options=format_options
|
||||
)
|
||||
```
|
||||
|
||||
## Utilities
|
||||
|
||||
### Progress Bar
|
||||
Visual progress tracking in console:
|
||||
|
||||
```python
|
||||
from utils.progress_bar import ProgressBar
|
||||
|
||||
# Create progress bar
|
||||
progress = ProgressBar(
|
||||
total=100,
|
||||
prefix="Processing:",
|
||||
suffix="Complete"
|
||||
)
|
||||
|
||||
# Update progress
|
||||
progress.start()
|
||||
for i in range(100):
|
||||
progress.increment()
|
||||
progress.finish()
|
||||
```
|
||||
|
||||
### Logging
|
||||
Centralized logging with file and GUI output:
|
||||
|
||||
```python
|
||||
from utils.logging_manager import LoggingManager
|
||||
|
||||
# Initialize logging
|
||||
logger = LoggingManager("./logs")
|
||||
|
||||
# Log messages
|
||||
logger.logger.info("Process started")
|
||||
logger.logger.error("An error occurred")
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **API Keys**
|
||||
- Never hardcode API keys
|
||||
- Use environment variables or the APIKeyManager
|
||||
- Store sensitive data securely
|
||||
|
||||
2. **Error Handling**
|
||||
- Always use try-except blocks for external services
|
||||
- Implement retries for network operations
|
||||
- Log errors appropriately
|
||||
|
||||
3. **Resource Management**
|
||||
- Use context managers for file operations
|
||||
- Close connections and handlers properly
|
||||
- Clean up temporary files
|
||||
|
||||
4. **Batch Processing**
|
||||
- Use appropriate batch sizes
|
||||
- Implement progress tracking
|
||||
- Handle partial failures gracefully
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### 1. File Access Errors
|
||||
```python
|
||||
# Use retry mechanism
|
||||
excel_service = ExcelService(max_retries=5)
|
||||
```
|
||||
|
||||
### 2. API Rate Limits
|
||||
```python
|
||||
# Configure batch size appropriately
|
||||
config = BatchConfig(batch_size=10, retry_delay=3)
|
||||
```
|
||||
|
||||
### 3. Memory Management
|
||||
```python
|
||||
# Process large files in chunks
|
||||
for chunk in pd.read_excel("large.xlsx", chunksize=1000):
|
||||
process_chunk(chunk)
|
||||
```
|
||||
|
||||
## Contributing Guidelines
|
||||
|
||||
1. Follow PEP 8 style guide
|
||||
2. Add type hints to new functions
|
||||
3. Include docstrings for all modules and functions
|
||||
4. Add unit tests for new features
|
||||
5. Update documentation when adding features
|
||||
|
||||
## License
|
||||
|
||||
MIT License - Feel free to use and modify as needed.
|
||||
Loading…
Reference in New Issue