Replicate minio fs depreceation

minio migration depreceated replicate

In previous post we talked about minio depreceation of FS backend and some of us confused, as the minio docs pretty confusing.

Let clear up a bit, based the minio docs.

Before June 2022

  • single drive deployment -> FS mode (called standalone mode)

After June 2022

  • single drive deployment -> xl-single (called distributed mode)

And if your deployment at beginning is multidrive -> distibuted mode by default xl, (this ripped out code wont affect)

At RELEASE.2022-10-29T06-21-33Z ripped out fs mode, left minio deployment on single drive before June 2022 borked. And one the strategy is Don't Upgrade at least that is how developer suggest. So yeah, we can just wait in future, will the fs implemented back "likely wont happen" or "will be migration guide some day".

Replicate:

version: "3.8"
services:
  s3:
    image: quay.io/minio/minio:RELEASE.2022-05-26T05-48-41Z
    #image: quay.io/minio/minio:RELEASE.2022-10-29T06-21-33Z
    command: server /data --console-address ":9001"
    user: 1000:1000
    environment:
      - MINIO_ROOT_USER=minioadmin
      - MINIO_ROOT_PASSWORD=minioadmin
    #ports:
    #  - 9000:9000
    #  - 9001:9001
    volumes:
      - ./data:/data

Deploy minio with older than june release, im using RELEASE.2022-05-26T05-48-41Z it will default to FS mode, lookup to .minio.sys/format.json

{"version":"1","format":"fs","id":"61cccbcb-16ce-4b4d-8eac-2f154a795c6e","fs":{"version":"2"}}

And redeploy with RELEASE.2022-10-29T06-21-33Z

ERROR Unable to use the drive /data: Drive /data: found backend type fs, expected xl or xl-single: Invalid arguments specified

2022-11-02-100923_1892x131_scrot

New deploy using RELEASE.2022-10-29T06-21-33Z, .minio.sys/format.json will show

{"version":"1","format":"xl-single","id":"11d2b225-782c-411d-b3b6-212536ccbe79","xl":{"version":"3","this":"f2e61ee3-b5f8-470a-9e84-b98384b832d0","sets":[["f2e61ee3-b5f8-470a-9e84-b98384b832d0"]],"distributionAlgo":"SIPMOD+PARITY"}}%

That is how we can replicate it, after few days of issue talk on minio docs there is a guide from developer for correct migration. On minio docs PR , the already up, and i will just paste it here just a doc for me and maybe you that looking for one.

Taken from PR

Migrate from Gateway or Filesystem Mode

Background

The MinIO Gateway and the related filesystem mode entered a feature freeze in July 2020. In February 2022, MinIO announced the `deprecation of the MinIO Gateway Along with the deprecation announcement, MinIO also announced that the feature would be removed in six months time.

As of RELEASE.2022-10-29T06-21-33Z, the MinIO Gateway and the related filesystem mode code have been removed. Deployments still using the standalone or filesystem MinIO modes that upgrade to RELEASE.2022-10-29T06-21-33Z or later receive an error when attempting to start MinIO.

Overview

To upgrade to a RELEASE.2022-10-29T06-21-33Z or later, those who were using the standalone or filesystem deployment modes must create a new Single-Node Single-Drive <minio-snsd> deployment and migrate settings and content to the new deployment.

This document outlines the steps required to successfully launch and migrate to a new deployment.

Standalone/file system mode continues to work on any release up to and including RELEASE.2022-10-24T18-35-07Z. To continue using a standalone deployment, install that MinIO release or any earlier release.

Do not upgrade to any release later than RELEASE.2022-10-24T18-35-07Z.

Procedure

  • Retrieve the existing deployment's environment variables

    Use the mc admin config export command to retrieve the environment variables defined on the existing standalone MinIO deployment. 

    mc admin config export ALIAS > config.txt
    • Replace ALIAS with the alias used for the existing standalone deployment you are retrieving values from.

  • Create a new Single-Node Single-Drive MinIO deployment

    Refer to the documentation for step-by-step instructions <deploy-minio-standalone> for launching a new |SNSD| deployment.

    The location of the deployment can be any empty folder on the storage medium of your choice. A new folder on the same drive can work for the new deployment as long as the existing deployment is not on the root of a drive. If the existing standalone system points to the root of the drive, you must use a separate drive for the new deployment.

    Set the port to a custom point different than the existing standalone deployment.

  • Add an alias for the new deployment with mc alias set 

      mc alias set PATH ACCESSKEY SECRETKEY
    • Replace PATH with the IP address or hostname and port for the new deployment.
    • Replace ACCESSKEY and SECRETKEY with the credentials you used when creating the new deployment.

  • Import configuration from existing standalone deployment to new deployment

      mc admin config import ALIAS  <  config.txt
    • Replace ALIAS with the alias for the new deployment.

  • Restart the server for the new deployment 

      mc admin service restart ALIAS
    • Replace ALIAS with the alias for the new deployment.

  • Duplicate buckets from existing standalone deployment to new deployment

    Use mc ls with the --json flag to retrieve a list of the buckets that exist on the standalone deployment. Use the list to recreate the buckets on the new deployment. 

      mc ls ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

  • (Optional) Duplicate tiers from existing standalone deployment to new deployment

    Use mc admin tier ls with the --json flag to retrieve a list of the tiers that exist on the standalone deployment. 

      mc admin tier ls ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

    Use the list to recreate the tiers on the new deployment.

  • Duplicate policies from existing standalone deployment to new deployment Use mc admin policy list with the --json flag to retrieve a list of policies that exist on the standalone deployment. 

      mc admin policy list ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

    Use the list to recreate the policies on the new deployment.

  • Duplicate groups from existing standalone deployment to new deployment

    Use mc admin group list with the --json flag to retrieve a list of groups that exist on the standalone deployment.

      mc admin group list ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

    Use the list to recreate the groups on the new deployment.

  • Duplicate users from existing standalone deployment to new deployment

    Use mc admin user list with the --json flag to retrieve a list of users with access key, policy name, and status.

      mc admin user list ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

    Use the list to recreate the users on the new deployment.

    Note: You will need to define each user SECRETKEY on the new deployment. Make note of the SECRETKEY for each user to let them know their new credentials.

  • Duplicate service accounts from existing standalone deployment to new deployment

    Use mc admin user svcacct list with the --json flag to list existing service accounts on the standalone deployment.

      mc admin user svcacct list ALIAS --json
    • Replace ALIAS with the alias for the existing standalone deployment.

    Use the list to recreate the service accounts on the new deployment.

  • Use mc mirror with the ~mc mirror --preserve and ~mc mirror --watch flags on the standalone deployment to move objects to the new |SNSD| deployment

      mc mirror --preserve --watch SOURCE/BUCKET TARGET/BUCKET
    • Replace SOURCE/BUCKET with the alias and a bucket for the existing standalone deployment.
    • Replace TARGET with the alias and corresponding bucket for the new deployment.

  • After the initial mirror process completes, convert standalone deployment to be read only

    One options for doing this is to remove policies that provide write permissions from users or groups with `mc admin policy unset command.

      mc admin policy unset ALIAS POLICYNAME [user=USERNAME | group=GROUPNAME]
    • Replace ALIAS with the alias for the existing standalone deployment
    • Replace POLICYNAME with the name of a policy on the existing standalone deployment.
    • Replace either USERNAME or GROUPNAME with either the user or group assigned the policy.

    Use care to not remove permissions for the user running the mc mirror command.

  • Wait for mc mirror to complete for all buckets for any remaining operations.

  • Stop the server for both deployments.

  • Restart the new MinIO deployment with the ports used for the previous standalone deployment.

    Refer to step four in the deploy SNSD

  • Confirm the new deployment works as expected.

    Verify that users and service accounts have access to the buckets and objects as usual.

And thats it, pretty confusing at first, but pretty clear what to do. good luck migrating

sources:

Previous Post Next Post

Add a comment