Media and object-storage admin runbook: profiles, access, and lifecycle

Beginner: a profile is not a purpose

Configure profiles in the storage area under /console/setting. A profile contains provider, endpoint, bucket, credentials, path, access mode, and lifecycle. Purpose mapping selects a profile for compliance, generated, staging, message_assets, and base64_error_url. Sharing a bucket does not make those purposes one security boundary.

The site supports Aliyun OSS, standard S3-compatible storage, Storage Tong, and task-report uploads. message_assets must use a private-proxy profile and cannot use task-report uploads. base64_error_url accepts only standard S3 or Aliyun OSS. Every enabled profile must be complete.

Minimum secure configuration

Use a dedicated service identity with minimum bucket and prefix permissions. The endpoint must be a trusted HTTP or HTTPS host; prefer HTTPS in production. Use an internal endpoint only on a trusted same-region network. Public mode is for deliberately public results; compliance and message assets use private. Restrict CDN, custom-domain, and bucket CORS origins, methods, and caching separately.

Copy{
  "id": "generated-s3",
  "provider": "s3",
  "enabled": true,
  "region": "auto",
  "use_path_style": true,
  "endpoint": "https://s3-compatible.example",
  "bucket": "generated-assets",
  "path_template": "generated/${YYYY}/${MM}/${DD}/${model}-${hash8}",
  "access_mode": "public",
  "lifecycle_enabled": true,
  "expiration_days": 30
}

Responses never return the Access Key Secret or STS JWT Secret, only flags that a secret exists. Leaving a secret blank while editing the same profile preserves its old value; it does not provide a plaintext recovery path.

Test, save, and lifecycle sync

Use this sequence: POST /api/storage/profiles/{id}/test against the proposed profile, PUT /api/storage/profiles to atomically save profiles and purpose mappings, then POST /api/storage/lifecycle/sync and verify the provider-side rule. All three require an operator identity and setting permission.

The test uploads and removes a small object, so even test credentials need scoped write and delete permission. Lifecycle sync preserves unrelated rules while updating this site's rule ID for a profile prefix. Confirm that the prefix cannot cover long-retention data. Provider expiry is usually asynchronous, so do not promise an exact deletion instant.

Public, private proxy, and signed URLs

public produces a directly reachable object URL; private should be delivered through site authorization. A signed URL is a time-limited bearer credential: anyone holding it can use its signed operation until expiry. Use short expiry, one object, and one method, and keep it out of Referrer data, analytics, chats, and error logs.

Remote upstream images are probed through an SSRF-protected client with metadata reads limited to 512 KiB. Operators still need trusted upstreams and egress policy. A channel's image_storage_enabled decides whether a result URL is recorded or proxied; it does not replace bucket authorization, moderation, or lifecycle.

Retention, cleanup, and database consistency

Initial defaults are public generated results for 30 days, public staging for 3 days, and private compliance data without automatic expiry. These are starting values, not legal advice. Message assets are automatically prepared as a separate private profile without lifecycle. Product, contract, and compliance owners approve actual retention.

Content-asset cleanup runs only on the master node and drains 50-row batches. Unbound temporary assets become eligible after 24 hours, unbound former assets after a one-hour grace, failed object deletion retries after 15 minutes, and one deletion times out after 30 seconds. Provider lifecycle is the storage-side backstop; database cleanup is the application consistency path. Monitor both.

Failure recovery and verification

For upload failures, separate DNS/TLS, endpoint, region, path style, credentials, bucket, KMS, CORS, and quota. A download 403 points to access mode, proxy authorization, signature expiry, or clock; a 404 points to object-key templates, lifecycle, or expired upstream results. Cleanup failure points to delete permission, object lock, or versioning.

Before changing endpoint, bucket, or purpose mapping, export configuration and object inventory. Probe read and write on a new profile before switching the purpose. Do not repoint an existing profile ID at an unrelated bucket because historical object references lose meaning. Drill credential rotation, bucket outage, lifecycle mistakes, missing objects after database restore, and stale CDN cache.

Expert: observability and audit

Measure upload success, first-byte and completion latency, bytes, delete failure, cleanup backlog, oldest orphan, lifecycle-sync result, 403/404, egress, and storage cost by profile and purpose. Log only profile ID, purpose, object-key digest, MIME, size, state, Request-ID, and task ID; exclude secrets, full signed URLs, Base64, and sensitive content.

Quarterly, review purpose mapping, least privilege, public access, CORS, lifecycle, versioning, object lock, encryption, backup, and recovery points. Sample generated-object reads and reconcile database-to-storage orphans in both directions. Validate deletion-policy changes on a test prefix before expansion.