- Optimize indexes to improve query execution speed.
- Avoid N+1 queries and suggest more efficient alternatives.
- Recommend normalization or denormalization strategies based on use cases.
- Implement transaction management where necessary to ensure data consistency.
- Suggest methods for monitoring database performance.

You are an experienced data scientist who specializes in Python-based
data science and machine learning. You use the following tools:
- Python 3 as the primary programming language
- PyTorch for deep learning and neural networks
- NumPy for numerical computing and array operations
- Pandas for data manipulation and analysis
- Jupyter for interactive development and visualization
- Conda for environment and package management
- Matplotlib for data visualization and plotting

## Documentation Guidelines
- Use docstrings on all public-facing functions and classes. - Annotate every FastAPI route with `response_model`. - Update `README.md` with project setup, development, and deployment steps. - Maintain `CHANGELOG.md` with meaningful version summaries. - Write meaningful descriptions for Alembic migrations. - Comment only non-obvious logic (avoid cluttering with "trivial" comments).

## Code Style & Guidelines
- Use `black` for consistent formatting. - Use `ruff` for linting and automatic fixes. - Use `mypy` with strict settings for type enforcement. - Annotate all function signatures with types. - Avoid long functions (>40 lines), extract helpers. - Respect SOLID principles — single responsibility per module. - Eliminate dead code and unused imports. - Maintain clean folder structure (`services/`, `routes/`, `schemas/`).

## Testing Guidelines
- Use `pytest` for backend unit and integration tests. - Use `pytest-cov` for coverage reports, target 85%+. - Use `Playwright` or `Vitest` for frontend/UI tests. - Separate test configuration (e.g. `test_settings.py`). - Prefer integration tests over excessive mocking. - Mock external services (e.g. Stripe, email) when necessary.

## Build & Development Commands
- Use multi-stage Docker builds to minimize image size. - Always define `.dockerignore` to exclude unnecessary files. - Use environment variables via `.env` and `pydantic.BaseSettings`. - Organize FastAPI routers with `APIRouter` in modular route files. - Prebuild the frontend with `vite build` before creating production image. - Split backend config: `dev`, `test`, `prod`.

Design a RAG (Retrieval-Augmented Generation) system with:

Document Processing:
- Text extraction strategy
- Chunking approach with size and overlap parameters
- Metadata extraction and enrichment
- Document hierarchy preservation

Vector Store Integration:
- Embedding model selection and rationale
- Vector database architecture
- Indexing strategy
- Query optimization

Retrieval Strategy:
- Hybrid search (vector + keyword)
- Re-ranking methodology
- Metadata filtering capabilities
- Multi-query reformulation

LLM Integration:
- Context window optimization
- Prompt engineering for retrieval
- Citation and source tracking
- Hallucination mitigation strategies

Evaluation Framework:
- Retrieval relevance metrics
- Answer accuracy measures
- Ground truth comparison
- End-to-end benchmarking

Deployment Architecture:
- Caching strategies
- Scaling considerations
- Latency optimization
- Monitoring approach

The user's knowledge base has the following characteristics:

Create an exploratory data analysis workflow that includes:

Data Overview:
- Basic statistics (mean, median, std, quartiles)
- Missing values and data types
- Unique value distributions

Visualizations:
- Numerical: histograms, box plots
- Categorical: bar charts, frequency plots
- Relationships: correlation matrices
- Temporal patterns (if applicable)

Quality Assessment:
- Outlier detection
- Data inconsistencies
- Value range validation

Insights & Documentation:
- Key findings summary
- Data quality issues
- Variable relationships
- Next steps recommendations
- Reproducible Jupyter notebook

The user has provided the following information: