Kangry.com [insert cool graphic here]
home | Topics | Logout | Search | Contact | ?? Kangry ?? | Bandwitdh
Topics:
DVR
nvrec
Mplayer
Links
Misc
Commands
Humor

Name

Password

New user

uploaded files
(misc)-> (Parent)->perl upload file via wget post (upload_to_kangry.pl) (with login) submited by Russell Wed 15 Apr 09
Edited Sat 31 Mar 12
Web kangry.com
It turns out that uploading files to a webserver is much more complicated than it should be. I wanted to create a simple and generic way to do this because I wanted a simple script that would run on just about any linux system. I didn't want to require any unusual library that isn't going to already be installed.

Here is what I ended up with: (full raw listing below)
#!/usr/bin/perl

$host='NAME_OF_COMPUTER';
$username="USERNAME";
$password="PASSWORD";

$MAX_SIZE=1000000;
$cookiefile="/tmp/kangry_cookies.txt";
$rawfile="/tmp/kangry_rawfile";
The first few lines set some specific settings. you will need to change the values for username and password. $cookiefile and $rawfile are tempoary files used by the program. the cookie file stores your login credentials. the Raw file is used to store the urlencoded version of each file that is uploaded. Maxsize could be increased but the webserver won't accept files over 2Mb (encoded, this could be increased)
print `wget -q --save-cookies $cookiefile --post-data 'name=$username&password=$password' -O /dev/null http://www.kangry.com/login.php`;
This line is essentially right out of the wget man page. It does the login, saving the cookie returned in the cookiefile. We don't actually care about the output of this request, so the results are routed to /dev/null
for ($n=0; $n<(@ARGV+0);$n++)
  {
  $name=$ARGV[$n];

  if (-f $name)
   {
      open(IN, $name);
      open (OUT,">$rawfile");

      ##read FILEHANDLE, $VAR, LENGTH [ , OFFSET ]
      $size=read IN,$RAW,$MAX_SIZE;
      print "uploading $name ($size bytes)\\n";
For all arguments, if the argument is an existing file , open that file and open the rawfile as a workspace. Then read up to $MAX_SIZE bytes.
     
      $RAW =~ s/([^A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg;
I hate this line. It's a bit of regexp black magic that I found here. This one line of code converts all the forbidden characters into %xy format. (%20 for a space etc ) On the one hand It's cool that it can be done in one line. On the other hand, I've stared at it for some time and I only kinda-understand how it works.
      print OUT "name=$name&size=$size&type=binary/octet-stream&host=$host&file=";
      print OUT $RAW;

     close OUT;
     close IN;
Into the working file, go a bunch of prefix data, telling the webserver than filename, size and type of the file. ( I could call file on the parameter, to get a more authentic type, but setting them all to binary works well enough ) The last term file= sets up the file for the raw data to follow that point. Then we print the urlencoded data into the file and we are done.
    print `wget  -q --load-cookies $cookiefile --post-file=$rawfile  -O /dev/null  http://www.kangry.com/files/upload.php`;
The other call to wget. This one uses the saved credentials in the cookie file to authenticate with the webserver and sends the contents of the rawfile to the webserver. I wanted to be able to call this command as a piped output, but this wouldn't work, I think because wget needs to know the length of the file before it begins.

I like that this program is pretty short and to the point. It uses wget to upload the files, but what it doesn't do is transmit the files the same way a real webbrowser does where it sends a multipart mime file as the posted data. I burned a lot of time trying to get that to work with wget. I was not successful.

I tried to implment the upload format that is shown here but it didn't work. I even got down to capturing packets to find the differences. For one thing, wget seems to put the upload into a second packet. I don't know if that was the problem. I gave up on that and switched to the uuencode method. The problem, is this method cannot be used to upload a file to an existing "file upload" web page. That's probably for the best, as it would be a easy way to fill somebody's file poster with spam.

What I did works, The cgi program ( be it perl, php or whatever) gets parameters for the files name and size. The parameter "file" contains the actual contents of the file (urlencoded). if your using PHP this is decoded for you in the $_REQUEST['file'] variable. I intend to post the server side of this as well (upload.php), but I'll need to clean it up a bit.

Full Listing:
#!/usr/bin/perl

$host='NAME_OF_COMPUTER';
$username="USERNAME";
$password="PASSWORD";

$MAX_SIZE=1000000;
$cookiefile="/tmp/kangry_cookies.txt";
$rawfile="/tmp/kangry_rawfile";

print `wget -q --save-cookies $cookiefile --post-data 'name=$username&password=$password'  -O /dev/null  http://www.kangry.com/login.php`;


for ($n=0; $n<(@ARGV+0);$n++)
  {
  $name=$ARGV[$n];

  if (-f $name)
   {
      open(IN, $name);
      open (OUT,">$rawfile");

      ##read FILEHANDLE, $VAR, LENGTH [ , OFFSET ]
      $size=read IN,$RAW,$MAX_SIZE;
      print "uploading $name ($size bytes)\\n";

      ## URLENCODE from http://support.internetconnection.net/CODE_LIBRARY/Perl_URL_Encode_and_Decode.shtml
      $RAW =~ s/([^A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg;

      print OUT "name=$name&size=$size&type=binary/octet-stream&host=$host&file=";
      print OUT $RAW;

     close OUT;
     close IN;
     print `wget  -q --load-cookies $cookiefile --post-file=$rawfile  -O /dev/null  http://www.kangry.com/files/upload.php`;
   }; # file exists

  }; ## for all arguments 

`rm $cookiefile`;
`rm $rawfile`;

View Download

On the server side:
<?php

{security code to make sure the login is valid}

    $filetype= $_REQUEST["type"];
    $size=$_REQUEST["size"];
    $filename=$_REQUEST["name"];
    $host=$_REQUEST["host"];
    $image=$_REQUEST["file"];
.....
At this point the variable $image contains the decoded image file as uploaded via the http post command. PHP un-escapes all that ugly %% notation for us. My code follows this with re-encoding to put it into a sql database. ( A binary file might easily have a quote or blackslash in it that will break a sql query. )

5/29/09
I just made a small change to the server side of this program. Now If the Host is set to "public" the file automatically becomes available in the public files list. This is a nice quick and dirty way to upload any file for hosting on the web from the command line.

7/8/11
As a commenter noted, in windows where there is difference in how text and binary files are handled, you need to add "binmode(IN);" just after " open(IN, $name);" or binary files will get corrupted. This is not an issue in linux.


Replys:
Re: perl upload file via wget (Anonymous)

Add comment or question...:
Subject:
Submited by: NOT email address. Leave blank for anonymous    (Spam Policy)

Enter Text: (text must match image for posting)




This file (the script that presented the data, not the data itself) , last modified Wednesday 03rd of December 2014 09:40:39 AM
your client: CCBot/2.0 (http://commoncrawl.org/faq/)
current time: Sunday 22nd of October 2017 07:39:25 AM