Last revision: 11 June 2021
Applicable version: 8.5.2 or above
v6 to v8 Backup Set Data Migration
Supports Debian 9 and 10 Installation
Supports macOS Big Sur (version 11) on Intel and ARM M1 CPU
Change Mobile Backup Destination
This feature will be of interest to partners who are planning to upgrade existing AhsayOBS v6 servers to AhsayCBS v8 or who have already upgraded from AhsayOBS v6 to AhsayCBS v8 and have not yet started or completed v6 to v8 backup set data migration, the conversion of v6 backup set data format to v8, 16 MB or 32 MB blocks format. For more details on v8 data format and structure, please refer to the following article, FAQ: Frequently Asked Questions about Ahsay Cloud Backup Suite 8.
With the release of AhsayCBS v8.5.2.0 and above, the v6 to v8 backup set data migration has been enhanced to run in a separate process, it was originally included in the AhsayCBS Java process. This means the data migration job will not utilize the Java memory resources of AhsayCBS Java process, as the data migration thread will run in its own independent Java process with its own dedicated Java memory setting. The backup set data migration job will not affect the performance or stability of the AhsayCBS service, even if the data migration job crashes. In the event the data migration job where to hang or become unresponsive the system administrator can terminate the process from the operating system.
As it is now a separate process, it is easier to allocate resources and to monitor the status of the data migration job. By default, the memory for the data migration is set to 1024 MB Java memory. However, the Java memory allocation can be customized by the system administrator.
The status of the data migration process can only be monitored from the operating system on the AhsayCBS server.
This illustrates how to read the information returned by the data migration process from the AhsayCBS server operating system. It is breakdown to the important factors such as the Java heap size, how to determine if it is a data migration process, the backup set ID used, how to know if it is an AhsayOBM or AhsayACB backup set, and the port used.
Although the example is from AhsayCBS data migration process in Windows, the general layout is similar for Linux, FreeBSD, and AhsayUBS.
| minimum Java heap size | This is the minimum Java heap size setting of the data migration process. The default value is 128 MB. |
|---|---|
| maximum Java heap size | This is the maximum Java heap size setting of the data migration process. The default value is 1024 MB. |
| Data Migration Process | This identifies the process as the data migration process. |
| Backup Set ID | The backup set ID of the backup set currently undergoing data migration. |
| Client Type | • OBM for AhsayOBM backup set • ACB for AhsayACB backup set |
| Port | The default port for migration job is 8081, if this port is taken then AhsayCBS will automatically try the next port 8082 and so on until a free port is found. |
In AhsayCBS for Windows, the following command can be used to check the Data Migration process: wmic process where caption=”java.exe” get commandline.
| C:\>wmic process where caption=“java.exe” get commandline CommandLine “C:\Program Files\AhsayCBS\java\bin\java” -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path=lib/WinX64 -cp “C:\Program Files\AhsayCBS\lib\cbs.jar;C:\Program Files\AhsayCBS\lib\*;C:\Program Files\AhsayCBS\lib;C:\Program Files\AhsayCBS\tomcat\lib\*” “-Dcatalina.base=C:\Program Files\AhsayCBS” RunDataMigration “C:\Program Files\AhsayCBS” 1620111338046 QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0u1zfJhC2cJPReetUm29xImSqBd2LJatigmkS3ZZgDp5J/D244eThpQ7xlZWwQrXGc= QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0te9GJsuIFiLYAh8wprdjBs QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0uYKdVll6zlR18edTvrKjuPskU4oKMpOdd1dcnn7kqh8fMSJwRHFGbl4Rm/GpS6aKfMS1TYGGyhS4ppp6s3d4eXum/UuA6d7/4Xt2vdWKlsvg== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0vC/gEJqRt1LFZr1Iz3nkhpvKKt6o7TuA+cseUlimZXV4L4YYYN3w+0qYiyBDCM32Q= OBM 127.0.0.1 8081 |
In AhsayCBS for Linux you can use the ps -ef|grep java command to list all the processes using java to run. This also starts with the username then the process ID to identify the running processes. In the example below, username root with process ID 10370 is the Data Migration process. This is indicated by the min/max java heap size -Xms128m -Xmx1024m.
| #root@centos-7-1810-temp ~]# ps -ef|grep java root 4905 1 0 17:12 ? 00:00:59 /usr/local/obm/jvm/bin/bschJW -Xms128m -Xmx768m -Dsun.nio.PageAlignDirectMemory=true -Djava.library.path=. -cp .:./cbs.jar cbs /usr/local/obm root 9796 1 18 20:40 pts/0 00:01:41 /usr/local/cbs/java/bin/java -Djava.util.logging.config.file=/usr/local/cbs/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.library.path=/usr/local/cbs/lib/LinX64 -DCATALINA_PID=/var/run/obsr.pid -Xrs -Xms512m -Xmx2048m -Xss384k -XX:MetaspaceSize=96m -XX:MaxMetaspaceSize=256m -XX:MaxDirectMemorySize=512m -XX:NewRatio=3 -XX:SurvivorRatio=32 -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=80 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=85 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -Dsun.net.inetaddr.ttl=3600 -Dnetworkaddress.cache.ttl=3600 -Dsun.net.inetaddr.negative.ttl=300 -Dnetworkaddress.cache.negative.ttl=300 -Dsun.nio.PageAlignDirectMemory=true -Djava.net.preferIPv4Stack=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=conf/logging.properties -Dtc.work.path=work -Dtc.log.path=logs -Djavax.servlet.context.tempdir=work -Djdk.nio.maxCachedBufferSize=262144 -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dignore.endorsed.dirs= -classpath /usr/local/cbs/tomcat/bin/bootstrap.jar:/usr/local/cbs/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/cbs -Dcatalina.home=/usr/local/cbs/tomcat -Djava.io.tmpdir=/usr/local/cbs/temp org.apache.catalina.startup.Bootstrap start root 10370 9796 99 20:49 pts/0 00:00:04 /usr/local/cbs/java-linux-x64/bin/java -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path=/usr/local/cbs/lib/LinX64 -cp /usr/local/cbs/lib/cbs.jar:/usr/local/cbs/lib/*:/usr/local/cbs/lib:/usr/local/cbs/tomcat/lib/* -Dcatalina.base=/usr/local/cbs RunDataMigration /usr/local/cbs 1620111338046 QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0uvzIAFFlBJq1ZNFwlLKxfO3B8uWZiGl35vDj0DjScn8Q== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0te9GJsuIFiLYAh8wprdjBs QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0sJtzhqrGM6IaxlRIL6+zjurGlJSNuslsqxg9vhbOvlNcM3Z9EaCwb7vMjHdMjKzVHxl/9zRmKcsCfovDWxFj8wa+yruhL0RYuOKuDdEqxcAw== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0vC/gEJqRt1LFZr1Iz3nkhpvKKt6o7TuA+cseUlimZXV4L4YYYN3w+0qYiyBDCM32Q= OBM 127.0.0.1 8081 root 10397 9112 0 20:49 pts/0 00:00:00 grep –color=auto java |
In AhsayCBS for FreeBSD, there are two steps to check the Data Migration process.
1. First find the process ID using the ps –aux|grep java command. To identify each process that is running, it also starts with the username then the process ID. In the example below, username root with process ID 2706 is the Data Migration process. This is indicated by the min/max java heap size -Xms128m -Xmx1024m.
| #root@FREEBSD120X86:~ # ps -aux|grep java root 2163 3.0 21.2 2635284 660996 0 I 22:21 3:25.68 /usr/local/cbs/java/bin/java -Djava.util.logging.config.file=/usr/local/cbs/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -D root 2706 0.0 0.9 1389572 27880 0 S 22:38 0:00.36 /usr/local/cbs/jvm/jre/bin/java -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path=/usr/local/cbs/lib/FbdX86 root 2708 0.0 0.1 4728 2228 0 S+ 22:38 0:00.00 grep java |
2. Then use procstat –c [process ID] to display the details of the Data Migration process.
| #root@FREEBSD120X86:~ # procstat -c 2706 PID COMM ARGS 2706 java /usr/local/cbs/jvm/jre/bin/java -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path=/usr/local/cbs/lib/FbdX86 -cp /usr/local/cbs/lib/cbs.jar:/usr/local/cbs/lib/*:/usr/local/cbs/lib:/usr/local/cbs/tomcat/lib/* -Dcatalina.base=/usr/local/cbs RunDataMigration /usr/local/cbs 1620111338046 QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0uvzIAFFlBJq1ZNFwlLKxfO3B8uWZiGl35vDj0DjScn8Q== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0te9GJsuIFiLYAh8wprdjBs QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0tWUU/zgGdsBufKrG40pssVYpVVpyQN1yLDxu2AXA2U5+uvTPSaYaMzdyCG0+rPhXZtzLZgkf5IfruVGGcXXMqtcosR9TVj+RLqnTaWA9mr5g== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0vC/gEJqRt1LFZr1Iz3nkhpvKKt6o7TuA+cseUlimZXV4L4YYYN3w+0qYiyBDCM32Q= OBM 127.0.0.1 8081 |
In AhsayUBS, there are two steps to check the Data Migration process.
1. First find the process ID using the ps –aux|grep java command. To identify each process that is running, it also starts with the username then the process ID. In the example below, username root with process ID 77431 is the Data Migration process. This is indicated by the min/max java heap size -Xms128m -Xmx1024m.
| ahsayubs:~ # ps -aux|grep java root 77431 58.3 3.6 2703736 291176 - S 03:39 0:16.85 /ubs/mnt/esfmfw/obsr/system/obsr/java-freebsd-x64/bin/java -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path root 45832 0.2 11.3 4044324 924100 - I 09:07 22:43.51 /ubs/module/obsr/system/obsr/java-freebsd-x64/bin/java -Djava.util.logging.config.file=/ubs/module/obsr/system/obsr/conf/logging.properties -Djava.util.logging.manager=o root 77434 0.0 0.0 6740 2624 0 S+ 03:39 0:00.00 grep java |
2. Then use procstat –c [process ID] to display the details of the Data Migration process.
| ahsayubs:~ # procstat -c 2706 PID COMM ARGS 77431 java /ubs/mnt/esfmfw/obsr/system/obsr/java-freebsd-x64/bin/java -Xms128m -Xmx1024m -XX:MaxDirectMemorySize=1024m -Dsun.nio.PageAlignDirectMemory=true -Xrs -Djava.library.path=/ubs/module/obsr/system/obsr/lib/FbdX64 -cp /ubs/mnt/esfmfw/obsr/system/obsr/lib/cbs.jar:/ubs/mnt/esfmfw/obsr/system/obsr/lib/*:/ubs/mnt/esfmfw/obsr/system/obsr/lib:/ubs/mnt/esfmfw/obsr/system/obsr/tomcat/lib/* -Dcatalina.base=/ubs/mnt/esfmfw/obsr/system/obsr RunDataMigration /ubs/mnt/esfmfw/obsr/system/obsr 1620111338046 QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0uPoAT6LpXuDZrAlUZ/Au6qhWmk1cZY7WQ2wa4GkkmyOLEpVjEsgNudF75MPRDHsl8= QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0te9GJsuIFiLYAh8wprdjBs QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0uA5iIH3nF6CztQiD4PrNEEyUcDFJJUeGMM5YVnjQ2uWiBAZZT2YIoe0v/Nop1mSOWRw+LgeNv5FV6+V4Pda1ffabgox4fsQbjALjCFwU6Oig== QgsSo/AkSjvTlOOVjcF/W9dcqxQKiFnVqI/A9Le/N0vC/gEJqRt1LFZr1Iz3nkhpvKKt6o7TuA+cseUlimZXV4L4YYYN3w+0qYiyBDCM32Q= OBM 127.0.0.1 8081 |
By default, the memory is set to 1024 MB. This can be changed by appending the options “com.ahsay.obs.core.job.MigrateV6Data.Xmx=xxxx” for the maximum Java heap size and “com.ahsay.obs.core.job.MigrateV6Data.Xms=xxxx” for the minimum Java heap size in the cbs.opt file.
If you wish to use the default Java heap size settings, then the additional options above are not required in the cbs.opt file.
To configure follow the steps below:
In AhsayCBS for Windows:
In AhsayCBS for Linux/FreeBSD:
1. Locate the cbs.opt file in /usr/local/cbs/conf folder.
| #root@centos-7-1810-temp ~]# cd /usr/local/cbs/conf [root@centos-7-1810-temp conf]# ls -la total 1468 drwxr-xr-x 4 root 4096 May 7 15:12 . drwxr-xr-x 23 root root 4096 Apr 30 17:45 .. -rwxr-xr-x 1 root root 316 Apr 30 17:44 acb-config.xml -rwxr-xr-x 1 root root 4961 Apr 30 17:44 afc.opt -rwxr-xr-x 1 root root 70 Apr 30 17:44 autoDiscovery.xml -rwxr-xr-x 1 root root 1024 Apr 30 17:44 autoUpdate.bdb -rwxr-xr-x 1 root root 1952 Apr 30 17:44 ca.crt drwxr-x— 3 root root 4096 Apr 30 17:44 Catalina -rwxr-xr-x 1 root root 12505 Apr 30 17:44 catalina.policy -rw-r–r– 1 root root 12315 May 1 20:46 catalina.properties -rwxr-xr-x 1 root root 12293 May 4 11:44 catalina.properties.1620108671165.bak -rw-r–r– 1 root root 12293 Apr 27 16:08 catalina.properties.1620186806193.bak -rwxr-xr-x 1 root root 30046 May 6 20:40 cbs.json -rwxr-xr-x 1 root root 30344 May 6 20:40 cbs.json.1 -rwxr-xr-x 1 root root 28385 May 4 14:11 cbs.json.10 -rwxr-xr-x 1 root root 28681 May 4 14:11 cbs.json.11 -rwxr-xr-x 1 root root 30344 May 6 20:40 cbs.json.2 -rwxr-xr-x 1 root root 29750 May 6 20:18 cbs.json.3 -rwxr-xr-x 1 root root 28084 May 6 20:18 cbs.json.4 -rwxr-xr-x 1 root root 28382 May 6 20:18 cbs.json.5 -rwxr-xr-x 1 root root 28385 May 5 12:00 cbs.json.6 -rwxr-xr-x 1 root root 28681 May 5 12:00 cbs.json.7 -rwxr-xr-x 1 root root 28086 May 5 11:52 cbs.json.8 -rwxr-xr-x 1 root root 28382 May 5 11:52 cbs.json.9 -rwxr-xr-x 1 root root 9618 Apr 30 17:44 cbs.opt |
2. Open the file, using a text editor add the options “com.ahsay.obs.core.job.MigrateV6Data.Xmx=value” and “com.ahsay.obs.core.job.MigrateV6Data.Xms=value” then save.
| com.ahsay.obs.core.job.MigrateV6Data.Xmx=2048 com.ahsay.obs.core.job.MigrateV6Data.Xms=128 |
3. Restart the AhsayCBS service.
| [root@centos-7-1810-temp conf]# cd /usr/local/cbs/bin [root@centos-7-1810-temp bin]# sh restart.sh |
In AhsayUBS:
1. Locate the cbs.opt file in /ubs/mnt/eslsfw/obsr/conf folder.
| # cd /ubs/mnt/eslsfw/obsr/conf # ls -la total 925 drwxr-xr-x 3 root wheel 142 Jul 9 02:15 . drwxr-xr-x 13 root wheel 13 Jul 8 08:25 .. -rwxr-xr-x 1 root wheel 316 Nov 19 2018 acb-config.xml -rwxr-xr-x 1 root wheel 4966 Jul 8 08:35 afc.opt -rwxr-xr-x 1 root wheel 70 Nov 19 2018 autoDiscovery.xml -rwxr-xr-x 1 root wheel 1152 Jul 8 08:48 autoUpdate.bdb -rwxr-xr-x 1 root wheel 1952 Nov 19 2018 ca.crt -rwxr-xr-x 1 root wheel 0 Nov 19 2018 Catalina -rwxr-xr-x 1 root wheel 12505 Nov 19 2018 catalina.policy -rwxr-xr-x 1 root wheel 8008 Nov 19 2018 catalina.properties -rwxr-xr-x 1 root wheel 19671 Jul 8 10:25 cbs.json -rwxr-xr-x 1 root wheel 19672 Jul 8 10:25 cbs.json.1 -rwxr-xr-x 1 root wheel 18755 Jul 8 08:43 cbs.json.10 -rwxr-xr-x 1 root wheel 19028 Jul 8 08:28 cbs.json.11 -rwxr-xr-x 1 root wheel 19672 Jul 8 09:09 cbs.json.2 -rwxr-xr-x 1 root wheel 19672 Jul 8 09:09 cbs.json.3 -rwxr-xr-x 1 root wheel 19373 Jul 8 08:46 cbs.json.4 -rwxr-xr-x 1 root wheel 19169 Jul 8 08:46 cbs.json.5 -rwxr-xr-x 1 root wheel 19169 Jul 8 08:44 cbs.json.6 -rwxr-xr-x 1 root wheel 19208 Jul 8 08:44 cbs.json.7 -rwxr-xr-x 1 root wheel 19208 Jul 8 08:43 cbs.json.8 -rwxr-xr-x 1 root wheel 18958 Jul 8 08:43 cbs.json.9 -rwxr-xr-x 1 root wheel 9666 Jul 8 08:54 cbs.opt |
2. Open the file, using a text editor add the options “com.ahsay.obs.core.job.MigrateV6Data.Xmx=value” and “com.ahsay.obs.core.job.MigrateV6Data.Xms=value” then save.
| com.ahsay.obs.core.job.MigrateV6Data.Xmx=2048 com.ahsay.obs.core.job.MigrateV6Data.Xms=128 |
3. Restart the AhsayCBS service.
| # cd /ubs/mnt/esfmfw/obsr/system/obsr/bin # sh restart.sh |
