Announcement

Collapse
No announcement yet.

Using the pre-commit hook to restrict the size of the submitted file failed

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using the pre-commit hook to restrict the size of the submitted file failed

    Dear Sir:
    How do you do?

    1.Environment description:
    Visual SVN Server 2.7.13, installed on Windows Server 2007 SP2 operating system.
    Now I create a version library named test. It works fine.

    2.My needs:
    I want to limit the size of the submitted files.

    3.My operation:
    1) I copy the file named pre-commit.tmpl and save it as file named pre-commit.sh in hooks directory.
    2) Modify the file named pre-commit.sh as follows:
    #!/bin/bash

    REPOS="$1"
    TXN="$2"

    SVNLOOK="C:/Program Files/VisualSVN Server/bin/svnlook"
    #SVNLOOK=/usr/bin/svnlook
    MAX_SIZE=512000

    files=$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}')

    # check check
    #if [[ $files =~ "project_nuli" ]];then
    for f in $files
    do
    # check file size
    filesize=$($SVNLOOK cat -t $TXN $REPOS $f | wc -c)
    if [ $filesize -gt $MAX_SIZE ] ; then
    echo "File $f is too large (must <= $MAX_SIZE)" >> /dev/stderr
    exit 1
    fi
    done
    #fi
    exit 0
    3) Restart the SVN service

    4.actual result:
    Then I submit a 46k file, but it succeeded. The new pre-commit hook seems not to work?

    What’s wrong?
    Fervently hope expert’s advice. Your early reply will be highly appreciated.

    Thank you so much!
    Yours
    yipianlvye
    June 30, 2018

  • #2
    On Linux the name of the file must be exactly "pre-commit" (not "pre-commit.sh"). I'm not an expert on Windows, but the extension that you use must be registered with Windows as an executable. If you've got ".sh" so registered then you get to move on to the next part of debugging: checking to see what happened. I normally do this by turning on the "shell echo" capability (e.g. "set -x") and then making sure that the output is redirected to some temporary file that I can evaluate later. In general, it might look like:

    set -x
    for x in 1; do
    put your code here
    done > /path/to/some/temp/file 2>&1

    Hopefully you'll then figure out what's going wrong.

    Comment


    • #3
      Thank you for your reply.
      I’ve solved this problem with the help of RD engineers. They found that the script could not run, mainly because the two commands (awk and wc) under Linux did not have corresponding commands under Windows. They rewritten the script as a Windows script and the problem was solved.

      Comment


      • #4
        Now I meet another question.
        Another repository named SW.
        URL: https://10.7.0.84:8443/svn/SW/软件事业部/01项目区/01工程项目/2017/甘肃掌上操作票专家系统项目/03.Code/3.2.Trunk/P70
        Checkout failed, Error: missing update-report close tag

        I execute ‘svnadmin verify E:/Repos/SW’ command on the server side, and the result is as follows:
        Verified revision 63848.
        svnadmin:E140001:zlib <uncompress>:corrupt dataecompression of svndiff data failed
        It seems that the 63849 version data has been corrupted.
        Version 63849 is as follows:
        [IMG]file:///C:%5CUsers%5Cyyc%5CAppData%5CLocal%5CTemp%5Cmsohtm lclip1%5C01%5Cclip_image002.jpg[/IMG]
        If the version's data is corrupted, what can we do to fix it or remove it completely? The svnadmin dump operation also failed.

        Comment


        • #5
          Use the dump and specify a range from 0-(N-1) and (N+1)-last. Then load up a new repo. This may not work if N+1 refers to N so you may need to iterate forward.

          Any working copy referring to any is the renumberred revisions will need to be discarded and re checked out (save the work outside the WC.

          Lot of pain. Repo corruption is very rare. You should look into why. Check windows error logs, etc. Prevent the issue for happening again.

          Comment


          • #6
            According to your method, the following commands are successful, and I get 3 dump files:
            svnadmin dump E:\Repos\SW -r 0:63848 > G:\SW1.dump
            svnadmin dump E:\Repos\SW -r 63850:66598 --incremental > G:\SW2.dump
            svnadmin dump E:\Repos\SW -r 66600:88013 --incremental > G:\SW3.dump
            But the execution of the following command meets an error:
            svnadmin load E:\Repos\SW1 < G:\SW1.dump

            Error information is below:
            ---Committed revision 2327 >>>

            <<< Started new transaction, based on original revision 2328
            * editing path:软件事业部/01项目区/05测试库/Project1 ...svnadmin:E200020
            :Invalid svn:mergeinfo value
            svnadmin:E200020:Unable to parse reversed revision range '96798-2321'
            svnadmin:E200020:Unable to parse reversed revision range '96798-2321'

            What’s wrong?
            Hope for your suggest. Thank you so much!

            Comment


            • #7
              That’s messed up meta-data. I’ve seen an old SVN bug that could have caused the issue, but I have no idea about Visual SVN. You should contact the Visual SVN folks for help.

              Comment


              • #8
                The script does not get the filenames which contain blank spaces. Anyone know how to read filenames with spaces? I tryed including " ", ' ', and \ before each space.
                For example:
                It get the size of "file.pdf", but nor for "file 1.pdf". I tryed including " " to the name, like "file .pdf", 'file .pdf', "file\ .pdf"

                Maybe SVNLOOK does not get filenames with spaces and there´s nothing to do...

                Comment


                • #9
                  Handling white-space within Linux scripting takes a lot of care in proper quoting. You have not shared the script so I cannot say where things went wrong.

                  Comment


                  • #10
                    That´s right, sorry! I forgot it

                    This is my script:
                    #!bin/bash
                    REPOS="$1"
                    TXN="$2"

                    SVNLOOK=/usr/bin/svnlook
                    MAX_SIZE=262144000
                    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4-
                    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
                    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed 's/ /\\\$/g' | tr $ " ")
                    files=$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}')
                    #echo "ARCHIVOS: $REPOS $TXN $files" >> /home/user/prueba.log
                    for f in $files
                    do
                    filesize=$($SVNLOOK cat -t $TXN $REPOS $f | wc -c)
                    #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
                    if [ $filesize -gt $MAX_SIZE ]; then
                    echo "File $f is too large, must be lower than 250MB">&2
                    exit 1
                    fi
                    done
                    exit 0


                    I tryed different ways of get results about SVNLOOK changed (commented options). As you see, I insert " " (and also can insert ' ' just changing it on my script). I also tryed with inserting \ before each space. Im checking the results of the script into log files, and I´m introducing " " and ' ' properly, but SVNLOOK cat does not get it...

                    Anyway, using the loop will be a problem if it check spaces, because it won´t get the proper file, it will get each word after each space as a different word.
                    Last edited by davidvazqueztds; 10-14-2019, 10:11 AM.

                    Comment


                    • #11
                      First, you could always use the "--xml" option to "svnlook" and then use an xml parser. You'd likely need to change languages away from bash.

                      Second, using "cut" is, well, tricky: I try to avoid it.

                      Third, because you're using "awk '{print $2}'" the data you're looking for is already broken since awk, by default, uses white-space to separate fields.

                      #!/bin/bash --noprofile
                      REPOS="$1"
                      TXN="$2"

                      SVNLOOK=/usr/bin/svnlook
                      MAX_SIZE=262144000
                      $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \
                      while read afile; do
                      case "$afile" in
                      */) continue;; # skip directories
                      esac
                      filesize=$($SVNLOOK cat -t "$TXN" "$REPOS" "$afile" | wc -c)
                      #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
                      if [[ $filesize -gt $MAX_SIZE ]]; then
                      echo "File $afile is too large, must be lower than 250MB">&2
                      exit 1
                      fi
                      done
                      exit 0

                      Give that a try (might be some syntax mistakes - I don't have time right now to test it.

                      Comment


                      • #12
                        I tryed it:

                        #!bin/bash
                        REPOS="$1"
                        TXN="$2"

                        SVNLOOK=/usr/bin/svnlook
                        MAX_SIZE=104857600 #250MB

                        files=$($SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//')
                        #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
                        #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'})
                        #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ")
                        echo "ARCHIVOS:$files" >> /home/user/prueba.log

                        #for f in $files
                        #do
                        while read files
                        do
                        case $files in */)
                        echo "CASE:$files" >> /home/user/prueba.log
                        continue
                        ;;
                        esac
                        filesize=$($SVNLOOK cat -t $TXN $REPOS $files | wc -c)
                        #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files)
                        #filesize=$(du -bs $files | cut -f 1)
                        echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
                        if [ $filesize -gt $MAX_SIZE ]; then
                        echo "SE CUMPLE" >> /home/user/prueba.log
                        echo "File $f is too large, must be lower than 250MB">&2
                        exit 1
                        fi
                        done
                        #done
                        exit 0

                        It does not work. The script does not reach the filesize function. In that case, is not getting the size of the file.
                        I don´t know why actually, I tryed it with filename without spaces and with spaces. No way to reach filesize... I think problem is in "while read", because if I use "for" instead, it works, but it does not go trhough case statment
                        any idea?

                        I think even if script works, the problem will be that SVNLOOK does not get file size if name contain spaces.
                        Last edited by davidvazqueztds; 10-17-2019, 07:43 AM.

                        Comment


                        • #13
                          You can't use an inline function like $() when you need to parse file names that have white-space.
                          So the "files=$()" is just plain not going to work.
                          In the code you posed up there's no $ in front of the "files" in "while read files". And you're not feeding anything into it so no, it's not going to work.

                          Did you try using the script as I posted it above? What happened?

                          Comment


                          • #14
                            Yes, I tryed your script but it does not work...
                            because "afile" variable is empty. In your code, you did not create the variable or save information inside, so code is not working.
                            Im doing it like you now:

                            #!bin/bash
                            REPOS="$1"
                            TXN="$2"

                            SVNLOOK=/usr/bin/svnlook
                            MAX_SIZE=104857600 #250MB
                            #files=$($SVNLOOK changed -t "$TXN" "$REPOS" | cut -d A -f2 | cut -d U -f2 | cut -c4-)
                            $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//'
                            #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
                            #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'})
                            #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ")
                            echo "ARCHIVOS:$files" >> /home/user/prueba.log

                            #for f in $files
                            #do
                            while read afile; do
                            echo "WHILE:$afile" >> /home/user/prueba.log
                            case $afile in */)
                            echo "CASE:$afile" >> /home/user/prueba.log
                            continue
                            ;;
                            esac
                            filesize=$($SVNLOOK cat -t $TXN $REPOS $afile | wc -c)
                            #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files)
                            #filesize=$(du -bs $files | cut -f 1)
                            echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
                            if [ $filesize -gt $MAX_SIZE ]; then
                            echo "SE CUMPLE" >> /home/user/prueba.log
                            echo "File $f is too large, must be lower than 250MB">&2
                            exit 1
                            fi
                            #done < "$files"
                            done
                            exit 0

                            Comment


                            • #15
                              In my script the line just before "while read afile; do" is:

                              $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \

                              That line ends in a pipe followed by a backslash which should then be followed by a newline (EOL).

                              Or, you can combine those 2 lines:

                              $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | while read afile; do

                              NOTE: the quoting for the sed commands should be apostrophe (single quotes) - not something silly that a copy/paste might turn them into. Same with the double-quotes.

                              Comment

                              Working...
                              X