Add design spec and implementation plan for restore-without-native-staging
Ignore .superpowers/ (subagent-driven-development's scratch progress ledger).
This commit is contained in:
Marek Miklewicz
@@ -0,0 +1,139 @@
|
||||
# Design: Restore Directly Into alt-mariadb Without Native Staging
|
||||
|
||||
Date: 2026-07-05
|
||||
Status: Approved by user, ready for implementation planning
|
||||
|
||||
## Goal
|
||||
|
||||
Make DirectAdmin account restores (both plugin-origin and external/foreign backups) land
|
||||
databases, database users, password hashes, and grants directly in `alt-mariadb`, without
|
||||
ever writing customer data into the CustomBuild-managed native MariaDB instance as an
|
||||
intermediate step.
|
||||
|
||||
## Background
|
||||
|
||||
The plugin already implements a "separated" architecture (see
|
||||
`mysql-installer/da-integration.md`): `alt-mariadb` is the real database service, CustomBuild's
|
||||
native MariaDB is kept installed only for DirectAdmin-native compatibility. Backup creation
|
||||
(`alt_mysql_user_backup_compress_pre.sh`) already augments every native DA backup with a
|
||||
`backup/alt_mysql/` payload containing the real `alt-mariadb` data, and restoring that payload
|
||||
(`alt_mysql_restore_payload.sh`) is already a direct, single-hop restore into `alt-mariadb`.
|
||||
|
||||
The remaining gap was restoring **external backups** — archives made on a server without this
|
||||
plugin, which therefore lack `backup/alt_mysql/manifest.json`. The existing fallback,
|
||||
`da_restore_native_staging_to_alt_mysql.sh`, assumes DirectAdmin has already restored the
|
||||
account's databases into the native MariaDB instance, then dumps and migrates them into
|
||||
`alt-mariadb`, then cleans up the native staged copies.
|
||||
|
||||
## Rejected alternative: point DirectAdmin at alt-mariadb directly
|
||||
|
||||
Early in this design process, a simpler-looking alternative was considered: install
|
||||
`alt-mariadb` on port 3306, rewrite DirectAdmin's own `/usr/local/directadmin/conf/mysql.conf`
|
||||
and `my.cnf` to point at it, and set `mysql_inst=no` in CustomBuild's `options.conf` so
|
||||
CustomBuild stops managing/version-gating the database. This was rejected:
|
||||
|
||||
- DirectAdmin's **legacy code-base license** enforces a hard version ceiling (MariaDB 10.6 /
|
||||
MySQL 8.0) at DA core login time, against whatever `mysql.conf` points to — independent of
|
||||
`mysql_inst`. This was confirmed by the user's own observed error: *"legacy code-base does
|
||||
not support this MySQL version, downgrade MySQL or upgrade DirectAdmin license."*
|
||||
`mysql_inst=no` only silences CustomBuild's own build-time/EOL-notification checks; it does
|
||||
not bypass this license gate.
|
||||
- `alt-mariadb` is installed specifically to run versions newer than what CustomBuild/legacy DA
|
||||
supports (10.11, 11.4, 11.8 — see `install_db.sh`). Pointing DA at it would either force
|
||||
capping `alt-mariadb` at 10.6 (defeating the point) or break DA login outright.
|
||||
- It provides no benefit the design below doesn't already achieve without that risk.
|
||||
|
||||
**Decision: no `INSTALL_MODE` toggle is added to `install_db.sh`. `alt-mariadb` is never made
|
||||
reachable through DirectAdmin's own `mysql.conf`/`my.cnf`, at any version.**
|
||||
|
||||
## Core design: self-service restore from the archive's native database files
|
||||
|
||||
Ground truth for DirectAdmin's native backup format was obtained by inspecting a real account
|
||||
backup (`example_backup/psy/backup/`), not from public docs (this format is not documented by
|
||||
DirectAdmin). For each native database, the backup contains a matching pair, flat under
|
||||
`backup/`:
|
||||
|
||||
- `backup/<dbname>.sql` — a plain `mariadb-dump`/`mysqldump` output (table structure + data).
|
||||
It contains **no `CREATE DATABASE`/`USE` statement** — the target database must already exist
|
||||
before this is imported.
|
||||
- `backup/<dbname>.conf` — a flat `key=value` text file:
|
||||
```
|
||||
accesshosts=0=localhost
|
||||
db_collation=DEFAULT_CHARACTER_SET_NAME=utf8mb3&DEFAULT_COLLATION_NAME=utf8mb3_general_ci&SCHEMA_NAME=<dbname>
|
||||
<username>=accesshosts=<host>&alter_priv=Y&...&passwd=%2A<HASH>&plugin=mysql_native_password&...&update_priv=Y
|
||||
```
|
||||
- Line 1: indexed list of database-level access hosts.
|
||||
- Line 2: `db_collation` — charset/collation needed to `CREATE DATABASE` correctly.
|
||||
- Line 3+: one line per MySQL user granted on this database, URL-encoded query-string
|
||||
fields, keyed by username. Contains the real `mysql_native_password` auth hash (not a
|
||||
plaintext password), the auth plugin, per-privilege `Y`/`N` flags (the same set of
|
||||
privilege columns `db_privilege_profile()` in `da_restore_native_staging_to_alt_mysql.sh`
|
||||
already reverse-engineers from `mysql.db`), and the granted host.
|
||||
|
||||
This pair is a complete, self-contained record: charset/collation, every user's exact password
|
||||
hash, full grant set, and access host(s) — sufficient to recreate the database and its users in
|
||||
`alt-mariadb` without DirectAdmin's cooperation or its native engine being involved at all.
|
||||
|
||||
### Restore flow
|
||||
|
||||
The post-restore hook (`alt_mysql_user_restore_post_pre_cleanup.sh`) already receives the
|
||||
original archive path (`$filename`/`$file`) and already checks for a plugin payload first. This
|
||||
design adds a second tier before falling back to native-staging-migrate:
|
||||
|
||||
1. **Plugin payload** (`backup/alt_mysql/manifest.json` present) — unchanged, existing direct
|
||||
restore via `alt_mysql_restore_payload.sh`.
|
||||
2. **Native `.conf`+`.sql` pairs present for a database matching the target user's prefix** —
|
||||
new: extract both files for that database directly from the archive, independent of
|
||||
whatever DirectAdmin itself did with its own native "database"/"database_data" restore
|
||||
items. Per database:
|
||||
- Parse `db_collation` → `CREATE DATABASE` with the correct charset/collation.
|
||||
- Parse each per-user line → `CREATE USER ... IDENTIFIED WITH mysql_native_password AS
|
||||
'<hash>'` at the parsed host, then translate the `Y`/`N` privilege flags into a `GRANT`
|
||||
statement (reuse/adapt the existing privilege-flag mapping from
|
||||
`db_privilege_profile()`/`rebuild_metadata_for_db()`).
|
||||
- Import the `.sql` dump into the created database.
|
||||
- Rebuild the plugin's ownership/grant-profile/access-host metadata tables, same as the
|
||||
existing native-staging path does today.
|
||||
- Verify the database exists and is non-empty before considering it restored, with the same
|
||||
rollback-dump-and-restore-on-failure safety already implemented in
|
||||
`alt_mysql_restore_payload.sh`/`da_restore_native_staging_to_alt_mysql.sh`.
|
||||
3. **Neither available for a given database** (malformed `.conf`, missing pair, or a genuinely
|
||||
foreign non-DA-native backup) — fall back to `da_restore_native_staging_to_alt_mysql.sh` for
|
||||
that database only, unchanged from today. This keeps the existing staging-then-migrate logic
|
||||
as a safety net, not the primary path.
|
||||
|
||||
Because the hook self-serves from the archive, **no restore-initiation wrapper or forced
|
||||
item-deselection (`select0=item`) is needed**. Whatever DirectAdmin does internally with its own
|
||||
native "database" restore item is irrelevant to this flow — if it stages a redundant copy into
|
||||
the native engine, that's harmless noise, not something the plugin needs to migrate from.
|
||||
|
||||
### What this simplifies going forward
|
||||
|
||||
- `da_restore_native_staging_to_alt_mysql.sh` becomes a per-database fallback, not the primary
|
||||
external-backup path. It is not removed.
|
||||
- CustomBuild's native MariaDB no longer needs to be usable as a restore staging target for the
|
||||
common case. It only needs to keep existing/running so DirectAdmin core's own login/version
|
||||
check has a legacy-safe target to talk to (per the rejected-alternative section above) — its
|
||||
role in `mysql-installer/da-integration.md` Phase 7 shrinks accordingly.
|
||||
- Backup creation (Phase 4) is unaffected.
|
||||
|
||||
## Risks / open items for implementation planning
|
||||
|
||||
- **The `.conf` format is not documented by DirectAdmin.** It was reverse-engineered from one
|
||||
real backup. Treat it defensively: validate structure before trusting it, and fail closed to
|
||||
the per-database native-staging fallback rather than assuming the format holds across
|
||||
DirectAdmin/CustomBuild versions.
|
||||
- Multiple users can be granted on one database (multiple `<username>=...` lines) — the parser
|
||||
must handle N users per `.conf`, not assume exactly one.
|
||||
- The exact temporary extraction directory/working path DirectAdmin uses during restore should
|
||||
still be logged and confirmed empirically on a disposable account as the first implementation
|
||||
task, consistent with how the backup-side hook was already built.
|
||||
- Password hashes are `mysql_native_password` in the observed example; the parser should not
|
||||
assume this is the only plugin value that can appear and should fail safely (fall back) for
|
||||
unrecognized auth plugins rather than guessing.
|
||||
|
||||
## Explicitly out of scope
|
||||
|
||||
- No `INSTALL_MODE=separated|simple` toggle in `install_db.sh`.
|
||||
- No changes to `install_db.sh`'s port/socket/datadir validation.
|
||||
- No rewriting of DirectAdmin's own `mysql.conf`/`my.cnf`.
|
||||
Reference in New Issue
Block a user