MySQL Server upgrade from ‘80034’ to ‘80035’ started.

The other day, the site went down because of a failed attempt to upgrade MySQL.

2023-11-13T09:34:21.867263Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.35-0ubuntu0.22.04.1) starting as process 103741
2023-11-13T09:34:21.881554Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-11-13T09:34:22.950026Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-11-13T09:34:26.678460Z 4 [System] [MY-013381] [Server] Server upgrade from '80034' to '80035' started.
2023-11-13T09:34:46.800661Z 4 [ERROR] [MY-013178] [Server] Execution of server-side SQL statement '-- Create general_log CREATE TABLE IF NOT EXISTS general_log (event_time TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6), user_host MEDIUMTEXT NOT NULL, thread_id BIGINT UNSIGNED NOT NULL, server_id INTEGER UNSIGNED NOT NULL, command_type VARCHAR(64) NOT NULL, argument MEDIUMBLOB NOT NULL) engine=CSV CHARACTER SET utf8mb3 comment="General log"; ' failed with error code = 13, error message = 'Can't get stat of './mysql/general_log.CSV' (OS errno 2 - No such file or directory)'.
2023-11-13T09:34:46.802289Z 0 [ERROR] [MY-013380] [Server] Failed to upgrade server.
2023-11-13T09:34:46.802414Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-11-13T09:34:47.696242Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.35-0ubuntu0.22.04.1)  (Ubuntu).

At first glance, it was quite self-explanatory – something was missing somewhere. The something part was obviously general_log.CSV and the somewhere part would seem to be the the MySQL datadir.

At this point, MySQL refused to start, but obtaining the datadir can be done by doing

sudo mysqld --verbose --help | grep ^datadir

In my case, which I guess is the default, it returned /var/lib/mysql/

The counter intuitive part, was that the missing file was actually supposed to go under there again, so fixing the hiccup was done by creating empty ones in a folder called mysql in the configured datadir.

sudo mkdir  /var/lib/mysql/mysql
sudo chown mysql:mysql /var/lib/mysql/mysql
sudo chmod 700 /var/lib/mysql/mysql

sudo touch /var/lib/mysql/mysql/general_log.CSV
sudo touch /var/lib/mysql/mysql/slow_log.CSV
sudo choown mysql:mysql /var/lib/mysql/mysql/general_log.CSV
sudo choown mysql:mysql /var/lib/mysql/mysql/slow_log.CSV

MySQL was now able to finish the upgrade and start.

2023-11-13T09:36:07.444532Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.35-0ubuntu0.22.04.1)  (Ubuntu).
2023-11-13T09:36:08.381336Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.35-0ubuntu0.22.04.1) starting as process 103939
2023-11-13T09:36:08.395700Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-11-13T09:36:09.466094Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-11-13T09:36:13.192365Z 4 [System] [MY-013381] [Server] Server upgrade from '80034' to '80035' started.
2023-11-13T09:38:00.596597Z 4 [System] [MY-013381] [Server] Server upgrade from '80034' to '80035' completed.