Announcement

Collapse
No announcement yet.

Using the pre-commit hook to restrict the size of the submitted file failed

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • davidvazqueztds
    replied
    Thank you very much! I should be care of that

    Leave a comment:


  • DougR
    replied
    filesize=$($SVNLOOK cat -t $TXN $REPOS $afile | wc -c)

    should be:

    filesize=$($SVNLOOK cat -t $TXN $REPOS "$afile" | wc -c)

    Those are double-quotes around the $afile without any white-space.

    Leave a comment:


  • davidvazqueztds
    replied
    Here is the result of the script (I also tryed once to introduce " " to the name, as you can see. Also with single quotes).
    Also tryed with the different ways of get file size (commented in the script).
    Last edited by davidvazqueztds; 10-18-2019, 10:35 AM.

    Leave a comment:


  • davidvazqueztds
    replied
    I tryed it. Now it react in different way.
    The problem is filesize function returns 0 value. I guess It happens because the name contain spaces, and the function "SVNLOOK cat -t $TXN $REPOS $afile | wc -c" canīt get the file size if name contains spaces.
    If name does not contain spaces, it works.
    Thats the thing now...

    Leave a comment:


  • DougR
    replied
    In my script the line just before "while read afile; do" is:

    $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \

    That line ends in a pipe followed by a backslash which should then be followed by a newline (EOL).

    Or, you can combine those 2 lines:

    $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | while read afile; do

    NOTE: the quoting for the sed commands should be apostrophe (single quotes) - not something silly that a copy/paste might turn them into. Same with the double-quotes.

    Leave a comment:


  • davidvazqueztds
    replied
    Yes, I tryed your script but it does not work...
    because "afile" variable is empty. In your code, you did not create the variable or save information inside, so code is not working.
    Im doing it like you now:

    #!bin/bash
    REPOS="$1"
    TXN="$2"

    SVNLOOK=/usr/bin/svnlook
    MAX_SIZE=104857600 #250MB
    #files=$($SVNLOOK changed -t "$TXN" "$REPOS" | cut -d A -f2 | cut -d U -f2 | cut -c4-)
    $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//'
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
    #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'})
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ")
    echo "ARCHIVOS:$files" >> /home/user/prueba.log

    #for f in $files
    #do
    while read afile; do
    echo "WHILE:$afile" >> /home/user/prueba.log
    case $afile in */)
    echo "CASE:$afile" >> /home/user/prueba.log
    continue
    ;;
    esac
    filesize=$($SVNLOOK cat -t $TXN $REPOS $afile | wc -c)
    #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files)
    #filesize=$(du -bs $files | cut -f 1)
    echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
    if [ $filesize -gt $MAX_SIZE ]; then
    echo "SE CUMPLE" >> /home/user/prueba.log
    echo "File $f is too large, must be lower than 250MB">&2
    exit 1
    fi
    #done < "$files"
    done
    exit 0

    Leave a comment:


  • DougR
    replied
    You can't use an inline function like $() when you need to parse file names that have white-space.
    So the "files=$()" is just plain not going to work.
    In the code you posed up there's no $ in front of the "files" in "while read files". And you're not feeding anything into it so no, it's not going to work.

    Did you try using the script as I posted it above? What happened?

    Leave a comment:


  • davidvazqueztds
    replied
    I tryed it:

    #!bin/bash
    REPOS="$1"
    TXN="$2"

    SVNLOOK=/usr/bin/svnlook
    MAX_SIZE=104857600 #250MB

    files=$($SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//')
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
    #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'})
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ")
    echo "ARCHIVOS:$files" >> /home/user/prueba.log

    #for f in $files
    #do
    while read files
    do
    case $files in */)
    echo "CASE:$files" >> /home/user/prueba.log
    continue
    ;;
    esac
    filesize=$($SVNLOOK cat -t $TXN $REPOS $files | wc -c)
    #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files)
    #filesize=$(du -bs $files | cut -f 1)
    echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
    if [ $filesize -gt $MAX_SIZE ]; then
    echo "SE CUMPLE" >> /home/user/prueba.log
    echo "File $f is too large, must be lower than 250MB">&2
    exit 1
    fi
    done
    #done
    exit 0

    It does not work. The script does not reach the filesize function. In that case, is not getting the size of the file.
    I donīt know why actually, I tryed it with filename without spaces and with spaces. No way to reach filesize... I think problem is in "while read", because if I use "for" instead, it works, but it does not go trhough case statment
    any idea?

    I think even if script works, the problem will be that SVNLOOK does not get file size if name contain spaces.
    Last edited by davidvazqueztds; 10-17-2019, 07:43 AM.

    Leave a comment:


  • DougR
    replied
    First, you could always use the "--xml" option to "svnlook" and then use an xml parser. You'd likely need to change languages away from bash.

    Second, using "cut" is, well, tricky: I try to avoid it.

    Third, because you're using "awk '{print $2}'" the data you're looking for is already broken since awk, by default, uses white-space to separate fields.

    #!/bin/bash --noprofile
    REPOS="$1"
    TXN="$2"

    SVNLOOK=/usr/bin/svnlook
    MAX_SIZE=262144000
    $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \
    while read afile; do
    case "$afile" in
    */) continue;; # skip directories
    esac
    filesize=$($SVNLOOK cat -t "$TXN" "$REPOS" "$afile" | wc -c)
    #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
    if [[ $filesize -gt $MAX_SIZE ]]; then
    echo "File $afile is too large, must be lower than 250MB">&2
    exit 1
    fi
    done
    exit 0

    Give that a try (might be some syntax mistakes - I don't have time right now to test it.

    Leave a comment:


  • davidvazqueztds
    replied
    Thatīs right, sorry! I forgot it

    This is my script:
    #!bin/bash
    REPOS="$1"
    TXN="$2"

    SVNLOOK=/usr/bin/svnlook
    MAX_SIZE=262144000
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4-
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/")
    #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed 's/ /\\\$/g' | tr $ " ")
    files=$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}')
    #echo "ARCHIVOS: $REPOS $TXN $files" >> /home/user/prueba.log
    for f in $files
    do
    filesize=$($SVNLOOK cat -t $TXN $REPOS $f | wc -c)
    #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log
    if [ $filesize -gt $MAX_SIZE ]; then
    echo "File $f is too large, must be lower than 250MB">&2
    exit 1
    fi
    done
    exit 0


    I tryed different ways of get results about SVNLOOK changed (commented options). As you see, I insert " " (and also can insert ' ' just changing it on my script). I also tryed with inserting \ before each space. Im checking the results of the script into log files, and Iīm introducing " " and ' ' properly, but SVNLOOK cat does not get it...

    Anyway, using the loop will be a problem if it check spaces, because it wonīt get the proper file, it will get each word after each space as a different word.
    Last edited by davidvazqueztds; 10-14-2019, 10:11 AM.

    Leave a comment:


  • DougR
    replied
    Handling white-space within Linux scripting takes a lot of care in proper quoting. You have not shared the script so I cannot say where things went wrong.

    Leave a comment:


  • davidvazqueztds
    replied
    The script does not get the filenames which contain blank spaces. Anyone know how to read filenames with spaces? I tryed including " ", ' ', and \ before each space.
    For example:
    It get the size of "file.pdf", but nor for "file 1.pdf". I tryed including " " to the name, like "file .pdf", 'file .pdf', "file\ .pdf"

    Maybe SVNLOOK does not get filenames with spaces and thereīs nothing to do...

    Leave a comment:


  • DougR
    replied
    That’s messed up meta-data. I’ve seen an old SVN bug that could have caused the issue, but I have no idea about Visual SVN. You should contact the Visual SVN folks for help.

    Leave a comment:


  • yipianlvye
    replied
    According to your method, the following commands are successful, and I get 3 dump files:
    svnadmin dump E:\Repos\SW -r 0:63848 > G:\SW1.dump
    svnadmin dump E:\Repos\SW -r 63850:66598 --incremental > G:\SW2.dump
    svnadmin dump E:\Repos\SW -r 66600:88013 --incremental > G:\SW3.dump
    But the execution of the following command meets an error:
    svnadmin load E:\Repos\SW1 < G:\SW1.dump

    Error information is below:
    ---Committed revision 2327 >>>

    <<< Started new transaction, based on original revision 2328
    * editing path:软件事业部/01项目区/05测试库/Project1 ...svnadmin:E200020
    :Invalid svn:mergeinfo value
    svnadmin:E200020:Unable to parse reversed revision range '96798-2321'
    svnadmin:E200020:Unable to parse reversed revision range '96798-2321'

    What’s wrong?
    Hope for your suggest. Thank you so much!

    Leave a comment:


  • DougR
    replied
    Use the dump and specify a range from 0-(N-1) and (N+1)-last. Then load up a new repo. This may not work if N+1 refers to N so you may need to iterate forward.

    Any working copy referring to any is the renumberred revisions will need to be discarded and re checked out (save the work outside the WC.

    Lot of pain. Repo corruption is very rare. You should look into why. Check windows error logs, etc. Prevent the issue for happening again.

    Leave a comment:

Working...
X