No announcement yet.

Assistance with a large repo migration to a new server!

  • Filter
  • Time
  • Show
Clear All
new posts

  • Assistance with a large repo migration to a new server!

    New to SVN. I need to migrate a 90GB repo to a new server (same OS). What would be my best approach?

    From online research looks like svndump and load is the best way to go, but 90GB would tale a long while to dump.

    Few questions:
    1) If I take the dump route, I cannot have the users stop access until I import. How will I import the changes made during the dump and load process? Is there a way to increment or copy only deltas after the initial main dump is collected?
    2) Is svnrdump the way to go?


  • #2
    If it were me, I'd make sure that the server was running SVN 1.10.latest (or 1.14.latest if delayed enough - should be out shortly) since those are long term support (LTS) releases.

    I would use "svnadmin dump /path/to/repo | ssh newserver svnadmin load /path/to/newrepo" (after creating the new repo using the newer SVN version) for the initial load as in this way the dump data will never have to be stored (I've seen dump data exceed the repository size by 20X and more).

    After that, I'd write a script to do incremental dump/load (actually, I already have but it belongs to my employer, WANdisco, Inc.) to "top up" the new repository from time to time. In that way the "switch-over" time can be relatively short.

    In general, I've not used "svnrdump" so can't say.


    • #3
      Thanks you so much for the response Doug! Really appreciate it.
      We are using version 1.9.7 dont think we will updating the SVN version before migrating to new server.

      We use Windows servers, What can be done for direct loads? "svnadmin dump /path/to/repo | ssh newserver svnadmin load /path/to/newrepo"
      And hooks/configs if any need to be moved manually right?
      I shall work on that incremental dump.
      Again, appreciate your help!


      • #4
        If you're already running SVN 1.9.x on your current serve and will install the same on the new server then no need to use dump/load - just use "rsync".

        However, realize that the Subversion community will be dropping support for 1.9.x as soon as it ships 1.14.0 so, if I were you, I'd run 1.10.x on the new server. And even if you do that the dump/load is not required because there's no real win to do so. So you'd still only need to use "rsync".



        • #5
          I looked up how to 'rsync' on windows servers.
          How will rsync affect the files currently in use? Or should we freeze the FS before running the rsync so that end users dont make changes?
          Keep in mind, its about 90GB.

          I'm running some tests now with dump and load. surprised load take way too long.


          • #6
            Given windows uses "hard locks" so I would be concerned that running "rsync" fully operational would cause "trouble". The "final rsync" should be done with the service down in any case. So you should likely just shut it down and do the rsync. Might even use "robocopy" if that's still around...

            If you're looking at reasonably fast machines and at least a 100MiB ethernet then 90GB will go very fast in the LAN. WAN is a different matter and will depend on latency and TCP drops.


            • #7
              Yes Doug, looking into robocopy and xcopy. Will keep you posted how that works out.
              Have a great weekend! cheers!

              P.S: For a later time, do you have experience/insight migrating SVN to Git?


              • #8
                In general, migrating SVN to Git is difficult - mostly because while SVN can and does tolerate binaries, Git has an extremely low tolerance.

                Example: 1TB SVN repository huge numbers of large binary images (executables, images, movies, zips, etc.) can checkout rather quickly since you only get a copy of a file if the revision being checked out needs it. Git, on the other hand, clones the entire repository - all 1TB of it.

                That's why Git has invested so much time in both Git LFS and "shallow clones".

                And, ultimately, the hardest part of a migration is the reorganization required to move those binaries into some other storage mechanism. And then fixing up all of the related build process.


                • #9
                  Hello Doug,

                  Appreciate all the insight. Been testing with robocopy and looks like thats the way we are going to go forward as the underlying SVN version is setup is same as our old server.
                  One thing that I have been thinking about is if a user has something checked out during the migration and commits it into the new SVN server, after the DNS change and all, will it affect anything?
                  Meaning will the user have issues? Or will it be transparent to them?


                  • #10
                    Check outs do not leave markers so it will be transparent.

                    "svn lock" does leave markers and the same exact account name must be used (or locks need to be cleared manually).


                    • #11
                      Appreciate all the insight. Will reachout if I hit any issues.