跳到主要内容

设计文件存储系统

问题​

如何用 Rust 设计一个支持分片上传的文件存储服务？

答案​

架构设计​

分片上传 API 实现​

use axum::{Router, Json, extract::{State, Path, Multipart}};
use uuid::Uuid;
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;

#[derive(Clone)]
struct UploadSession {
    file_name: String,
    total_parts: u32,
    uploaded_parts: HashMap<u32, String>, // part_number → 本地路径
}

type Sessions = Arc<RwLock<HashMap<String, UploadSession>>>;

/// 1. 创建上传会话
async fn create_upload(
    State(sessions): State<Sessions>,
    Json(req): Json<CreateUploadRequest>,
) -> Json<CreateUploadResponse> {
    let upload_id = Uuid::new_v4().to_string();
    let session = UploadSession {
        file_name: req.file_name,
        total_parts: req.total_parts,
        uploaded_parts: HashMap::new(),
    };
    sessions.write().await.insert(upload_id.clone(), session);
    Json(CreateUploadResponse { upload_id })
}

/// 2. 上传单个分片
async fn upload_part(
    State(sessions): State<Sessions>,
    Path((upload_id, part_number)): Path<(String, u32)>,
    mut multipart: Multipart,
) -> axum::http::StatusCode {
    if let Some(field) = multipart.next_field().await.unwrap() {
        let data = field.bytes().await.unwrap();
        let part_path = format!("/tmp/uploads/{}_{}", upload_id, part_number);
        tokio::fs::write(&part_path, &data).await.unwrap();

        let mut sessions = sessions.write().await;
        if let Some(session) = sessions.get_mut(&upload_id) {
            session.uploaded_parts.insert(part_number, part_path);
        }
    }
    axum::http::StatusCode::OK
}

/// 3. 合并分片
async fn complete_upload(
    State(sessions): State<Sessions>,
    Path(upload_id): Path<String>,
) -> Json<CompleteUploadResponse> {
    let sessions = sessions.read().await;
    let session = sessions.get(&upload_id).unwrap();

    let output_path = format!("/storage/{}", session.file_name);
    let mut output = tokio::fs::File::create(&output_path).await.unwrap();

    // 按 part_number 顺序合并
    for i in 0..session.total_parts {
        let part_path = &session.uploaded_parts[&i];
        let data = tokio::fs::read(part_path).await.unwrap();
        tokio::io::AsyncWriteExt::write_all(&mut output, &data).await.unwrap();
    }

    Json(CompleteUploadResponse { url: output_path })
}

核心设计要素​

要素设计
分片大小5-10MB（兼顾性能和重试成本）
秒传上传前计算文件 hash，已存在则跳过
断点续传记录已上传分片，客户端跳过已完成的
并发上传多个分片可并行上传
清理超时未完成的会话定期清理临时分片

常见面试问题​

Q1: 如何实现秒传？​

答案：

  1. 客户端计算文件的 SHA-256 Hash
  2. 上传前发送 Hash 到服务端查询
  3. 如果服务端已有相同 Hash 的文件，直接返回 URL（秒传成功）
  4. 否则正常走分片上传流程

关键：Hash 计算在客户端用 Web Worker 或 Rust Wasm。

相关链接​