mirror of
https://github.com/rnvm9wjdtj-bot/myaps_api.git
synced 2026-06-02 05:54:40 +00:00
fix(security,config): 移除默认高权限密码，统一数据库配置逻辑
- 删除core/settings.py中的默认高权限配置项 - 重构binlog监听工具的凭据获取逻辑，不再硬编码root账号与默认密码 - 重写项目README文档，更新配置说明与启动流程 - 新增API安全与稳定性修复计划文档，梳理后续优化方向
This commit is contained in:
@@ -1,62 +1,181 @@
|
||||
# MyAPS FastAPI Project
|
||||
# MyAPS API
|
||||
|
||||
A simple FastAPI project template with a Python virtual environment.
|
||||
MyAPS API 是一个基于 FastAPI 的企业级数据操作平台，包含数据接口、数据清洗、监控、WebSocket 通信、日志与定时任务等能力。
|
||||
|
||||
## Project Structure
|
||||
## 项目概览
|
||||
|
||||
```
|
||||
myaps_fastapi/
|
||||
├── venv/ # Python virtual environment
|
||||
├── main.py # Main FastAPI application
|
||||
├── requirements.txt # Project dependencies
|
||||
├── .gitignore # Git ignore file
|
||||
└── README.md # Project documentation
|
||||
### 目录结构
|
||||
|
||||
```text
|
||||
myaps_api/
|
||||
├── apps/ # 业务模块
|
||||
│ ├── common/ # 监控、帮助、通用工具
|
||||
│ ├── data_opt/ # 数据操作、清洗、binlog、调度
|
||||
│ └── io_api/ # 对外 API 接口
|
||||
├── core/ # 应用工厂、配置、数据库、生命周期
|
||||
├── globalobjects/ # 全局对象、日志、数据库管理
|
||||
├── static/ # 前端静态资源
|
||||
├── scripts/ # 开发、部署、迁移脚本
|
||||
├── tests/ # 自动化测试
|
||||
├── main.py # 应用入口
|
||||
├── requirements.txt # Python 依赖
|
||||
└── .env.example # 环境变量示例
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
### 技术栈
|
||||
|
||||
### 1. Activate the Virtual Environment
|
||||
- FastAPI
|
||||
- Uvicorn
|
||||
- Tortoise ORM
|
||||
- Pydantic
|
||||
- MySQL / PostgreSQL / SQLite
|
||||
- Redis
|
||||
|
||||
**Windows:**
|
||||
```cmd
|
||||
venv\Scripts\activate
|
||||
```
|
||||
## 快速开始
|
||||
|
||||
### 1. 准备虚拟环境
|
||||
|
||||
项目依赖当前已安装在仓库内虚拟环境 `venv` 中。
|
||||
|
||||
Linux / macOS:
|
||||
|
||||
**macOS/Linux:**
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
```
|
||||
|
||||
### 2. Install Dependencies
|
||||
或者直接使用虚拟环境解释器执行命令：
|
||||
|
||||
```cmd
|
||||
pip install -r requirements.txt
|
||||
```bash
|
||||
./venv/bin/python --version
|
||||
```
|
||||
|
||||
### 3. Run the FastAPI Server
|
||||
### 2. 准备环境变量
|
||||
|
||||
```cmd
|
||||
uvicorn main:app --reload
|
||||
复制示例文件并按实际环境填写：
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
### 4. Access the API
|
||||
至少需要确认以下配置：
|
||||
|
||||
- Open your browser and go to [http://127.0.0.1:8000](http://127.0.0.1:8000)
|
||||
- Swagger UI documentation: [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
|
||||
- ReDoc documentation: [http://127.0.0.1:8000/redoc](http://127.0.0.1:8000/redoc)
|
||||
```bash
|
||||
PROJECT_DIR=YOUR_PROJECT_DIR
|
||||
MYAPS_DB_HOST=localhost
|
||||
MYAPS_DB_PORT=3333
|
||||
MYAPS_DB_USER=your_db_user
|
||||
MYAPS_DB_PASSWORD=your_db_password
|
||||
MYAPS_DB_SET=db1,db2
|
||||
MYAPS_MAIN_DB=db1
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
如使用 PostgreSQL staging / 清洗能力，还需要补充：
|
||||
|
||||
- `GET /` - Root endpoint with welcome message
|
||||
- `GET /api/info` - Project information
|
||||
```bash
|
||||
THIS_DB_HOST=localhost
|
||||
THIS_DB_PORT=5432
|
||||
THIS_DB_USER=postgres
|
||||
THIS_DB_PASSWORD=your_password
|
||||
THIS_DB_NAME=appsmith
|
||||
```
|
||||
|
||||
## Features
|
||||
### 3. 启动服务
|
||||
|
||||
- FastAPI framework for high-performance API development
|
||||
- Automatic interactive API documentation
|
||||
- Python virtual environment for dependency isolation
|
||||
- Git integration with proper ignore rules
|
||||
使用项目脚本：
|
||||
|
||||
## Requirements
|
||||
```bash
|
||||
./scripts/dev_server.sh start
|
||||
```
|
||||
|
||||
- Python 3.7+","}}}
|
||||
直接启动：
|
||||
|
||||
```bash
|
||||
./venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 8000 --reload
|
||||
```
|
||||
|
||||
### 4. 访问服务
|
||||
|
||||
- 首页: `http://127.0.0.1:8000/`
|
||||
- Swagger 文档: `http://127.0.0.1:8000/docs`
|
||||
|
||||
## 常用命令
|
||||
|
||||
### 测试
|
||||
|
||||
运行全部测试：
|
||||
|
||||
```bash
|
||||
./venv/bin/python -m pytest tests/ -v
|
||||
```
|
||||
|
||||
验证应用可导入：
|
||||
|
||||
```bash
|
||||
./venv/bin/python -c "import main; print(main.app.title)"
|
||||
```
|
||||
|
||||
### 开发脚本
|
||||
|
||||
```bash
|
||||
./scripts/dev_server.sh start
|
||||
./scripts/dev_server.sh stop
|
||||
./scripts/dev_server.sh restart
|
||||
./scripts/dev_server.sh status
|
||||
./scripts/dev_server.sh logs
|
||||
```
|
||||
|
||||
## 数据库配置说明
|
||||
|
||||
### 全局数据库账号
|
||||
|
||||
项目统一使用以下变量作为主数据库连接配置：
|
||||
|
||||
```bash
|
||||
MYAPS_DB_HOST
|
||||
MYAPS_DB_PORT
|
||||
MYAPS_DB_USER
|
||||
MYAPS_DB_PASSWORD
|
||||
```
|
||||
|
||||
### binlog 配置说明
|
||||
|
||||
binlog 相关校验与参数设置逻辑当前也统一使用：
|
||||
|
||||
```bash
|
||||
MYAPS_DB_USER
|
||||
MYAPS_DB_PASSWORD
|
||||
```
|
||||
|
||||
注意事项：
|
||||
|
||||
- 不再使用单独的 `MYAPS_ROOT_PASSWORD`
|
||||
- binlog 相关逻辑不会再写死 `root` 用户
|
||||
- 若 `MYAPS_DB_USER` 或 `MYAPS_DB_PASSWORD` 未配置，相关校验会显式失败
|
||||
- 如需执行高权限 binlog 配置操作，请确保 `MYAPS_DB_USER` 对应账号本身具备所需权限
|
||||
|
||||
### staging / 清洗模式
|
||||
|
||||
- `THIS_DB_*` 用于 PostgreSQL staging 数据库配置
|
||||
- `STAGING_DB_NAME` 默认为 `--s`
|
||||
- 清洗模式与主业务数据库配置分离
|
||||
|
||||
## 监控与日志
|
||||
|
||||
- 实时日志与历史日志页面位于 `/monitor`
|
||||
- 统一日志系统位于 `globalobjects/logger/`
|
||||
- 开发期可通过 `./scripts/dev_server.sh logs -f` 查看实时日志
|
||||
|
||||
## 当前验证状态
|
||||
|
||||
在当前仓库环境下，以下命令已验证通过：
|
||||
|
||||
```bash
|
||||
./venv/bin/python -m pytest tests/ -v
|
||||
./venv/bin/python -c "import main; print(main.app.title)"
|
||||
```
|
||||
|
||||
## 备注
|
||||
|
||||
- 若 `PROJECT_DIR` 未配置，应用配置加载会失败
|
||||
- 若数据库或 Redis 未就绪，部分功能会在启动或运行阶段报出明确错误
|
||||
- 修改数据库、binlog、生命周期逻辑后，建议至少重新执行一次测试和导入校验
|
||||
|
||||
@@ -61,7 +61,7 @@ from pymysqlreplication.row_event import (
|
||||
DeleteRowsEvent,
|
||||
)
|
||||
|
||||
from core.settings import MYAPS_DB_HOST, MYAPS_DB_PORT, MYAPS_DB_USER, MYAPS_DB_PASSWORD, MYAPS_MAIN_DB, MYAPS_DBSET_LIST, TURNON_BINLOG_LISTENER, ENABLE_BINLOG_POSITION, MYAPS_ROOT_PASSWORD
|
||||
from core.settings import MYAPS_DB_HOST, MYAPS_DB_PORT, MYAPS_DB_USER, MYAPS_DB_PASSWORD, MYAPS_MAIN_DB, MYAPS_DBSET_LIST, TURNON_BINLOG_LISTENER, ENABLE_BINLOG_POSITION
|
||||
|
||||
from globalobjects import logger as log_config
|
||||
from globalobjects.reminder import remind_manager, RemindType
|
||||
@@ -180,6 +180,32 @@ LOG_LEVEL = os.getenv("LOG_LEVEL") or "INFO"
|
||||
logger = log_config.get_logger(__name__, level=LOG_LEVEL)
|
||||
|
||||
|
||||
def _get_binlog_db_credentials(action_name: str) -> Optional[Dict[str, Any]]:
|
||||
"""获取用于 binlog 高权限操作的数据库连接信息。"""
|
||||
db_user = MYAPS_DB_USER
|
||||
db_password = MYAPS_DB_PASSWORD
|
||||
if not db_user:
|
||||
logger.error(
|
||||
f"{action_name}失败: 未配置 MYAPS_DB_USER。"
|
||||
"请在 .env 或项目配置中显式提供该变量。"
|
||||
)
|
||||
return None
|
||||
|
||||
if not db_password:
|
||||
logger.error(
|
||||
f"{action_name}失败: 未配置 MYAPS_DB_PASSWORD。"
|
||||
"请在 .env 或项目配置中显式提供该变量。"
|
||||
)
|
||||
return None
|
||||
|
||||
return {
|
||||
"host": MYAPS_DB_HOST,
|
||||
"port": MYAPS_DB_PORT,
|
||||
"user": db_user,
|
||||
"password": db_password,
|
||||
}
|
||||
|
||||
|
||||
class BinlogPositionManager:
|
||||
"""Binlog 位置管理器 - 负责持久化和恢复 binlog 位置（基于文件存储）"""
|
||||
|
||||
@@ -2028,14 +2054,12 @@ def is_mysql_config_valid() -> bool:
|
||||
bool: 当所有配置项都符合要求时返回True，其他情况返回False
|
||||
"""
|
||||
|
||||
# 数据库连接信息
|
||||
db_host = MYAPS_DB_HOST
|
||||
db_port = MYAPS_DB_PORT
|
||||
db_user = "root"
|
||||
db_password = MYAPS_ROOT_PASSWORD
|
||||
credentials = _get_binlog_db_credentials("验证MySQL配置")
|
||||
if not credentials:
|
||||
return False
|
||||
|
||||
logger.info("🚀 开始验证MySQL配置...")
|
||||
logger.info(f"🔗 连接到数据库: {db_host}:{db_port}")
|
||||
logger.debug("🚀 开始验证MySQL配置...")
|
||||
logger.debug(f"🔗 连接到数据库: {credentials['host']}:{credentials['port']}")
|
||||
|
||||
var_result = {
|
||||
"log_bin": "ON",
|
||||
@@ -2047,10 +2071,10 @@ def is_mysql_config_valid() -> bool:
|
||||
try:
|
||||
# 连接数据库
|
||||
conn = pymysql.connect(
|
||||
host=db_host,
|
||||
port=int(db_port),
|
||||
user=db_user,
|
||||
password=db_password,
|
||||
host=credentials["host"],
|
||||
port=int(credentials["port"]),
|
||||
user=credentials["user"],
|
||||
password=credentials["password"],
|
||||
connect_timeout=5
|
||||
)
|
||||
|
||||
@@ -2087,22 +2111,20 @@ def set_binlog_params():
|
||||
3. 验证设置是否成功
|
||||
"""
|
||||
|
||||
# 数据库连接信息
|
||||
db_host = MYAPS_DB_HOST
|
||||
db_port = MYAPS_DB_PORT
|
||||
db_user = "root"
|
||||
db_password = MYAPS_ROOT_PASSWORD
|
||||
credentials = _get_binlog_db_credentials("设置binlog参数")
|
||||
if not credentials:
|
||||
exit(1)
|
||||
|
||||
logger.info("🚀 开始设置binlog参数...")
|
||||
logger.info(f"🔗 连接到数据库: {db_host}:{db_port}")
|
||||
logger.info(f"🔗 连接到数据库: {credentials['host']}:{credentials['port']}")
|
||||
|
||||
try:
|
||||
# 连接数据库
|
||||
conn = pymysql.connect(
|
||||
host=db_host,
|
||||
port=int(db_port),
|
||||
user=db_user,
|
||||
password=db_password,
|
||||
host=credentials["host"],
|
||||
port=int(credentials["port"]),
|
||||
user=credentials["user"],
|
||||
password=credentials["password"],
|
||||
connect_timeout=5
|
||||
)
|
||||
|
||||
|
||||
@@ -153,7 +153,6 @@ MYAPS_DB_HOST = os.getenv("MYAPS_DB_HOST") or json_env_config.get("MYAPS_DB_HOST
|
||||
MYAPS_DB_PORT = int(os.getenv("MYAPS_DB_PORT") or json_env_config.get("MYAPS_DB_PORT") or 3333)
|
||||
MYAPS_DB_USER = os.getenv("MYAPS_DB_USER") or json_env_config.get("MYAPS_DB_USER")
|
||||
MYAPS_DB_PASSWORD = os.getenv("MYAPS_DB_PASSWORD") or json_env_config.get("MYAPS_DB_PASSWORD")
|
||||
MYAPS_ROOT_PASSWORD = os.getenv("MYAPS_ROOT_PASSWORD") or json_env_config.get("MYAPS_ROOT_PASSWORD") or "E9damw0o@#"
|
||||
MYAPS_DB_SET = os.getenv("MYAPS_DB_SET") or json_env_config.get("MYAPS_DB_SET")
|
||||
if not MYAPS_DB_SET:
|
||||
logger.warning_msg("环境变量配置", "MYAPS_DB_SET 未设置")
|
||||
@@ -179,4 +178,3 @@ REDIS_HOST = os.getenv("REDIS_HOST") or json_env_config.get("REDIS_HOST") or "12
|
||||
REDIS_PORT = int(os.getenv("REDIS_PORT") or json_env_config.get("REDIS_PORT") or 6379)
|
||||
REDIS_DB = int(os.getenv("REDIS_DB") or json_env_config.get("REDIS_DB") or 0)
|
||||
REDIS_PASSWORD = os.getenv("REDIS_PASSWORD") or json_env_config.get("REDIS_PASSWORD") or ""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user