Downloading Files from Web using Perl - GeeksforGeeks

02 Feb, 2022
Perl is a multi-purpose interpreted language that is often implemented using Perl scripts that can be saved using the .pl extension and run directly using the terminal or command prompt. It is a stable, cross-platform language that was developed primarily with strong capabilities in terms of text manipulation and modifying, and extracting information from web pages. It is under active development and open source. It finds major use in web development, system administration, and even GUI development due to its capability of working with HTML, XML, and other mark-up languages. It is prominently used along with the Web as it can handle encrypted web data in addition to E-Commerce transactions. 
In this article, we will be seeing different approaches to download web pages as well as images using Perl scripts.
In this approach, we write a sub routine where a URL is passed to a system command. The variable stores the content of the web page in the raw HTML form. We then return these contents.
Perl
#!usr/bin/perl  # using the strict pragmause strict;  # using the warnings pragma# to generate warnings in case of incorrect# codeuse warnings;  # specifying the Perl version use 5.010;  # declaring the sub routinesub getWebPage {      # variable to store the URL    my $url = 'http://www.google.com/';          # variable to store the contents of the     # web page    my $webpage = system     "wget --output-document=- $url";          # returning the contents of the web page    return $webpage;}  # printing user friendly messagesay "the contents of the downloaded web page : ";  # calling the sub routinegetWebPage();
Output:
the contents of the downloaded web page :
<raw HTML web page>
This approach is exactly the same as above, the only difference being that here the system command used is “curl” in place of “wget”.
Perl
#!usr/bin/perl  # using the strict pragmause strict;  # using the warnings pragma to # generate warnings in case of # erroneous codeuse warnings;  # specifying the Perl versionuse 5.010;  # declaring the sub routinesub getWebPage {      # variable to store the URL    my $url = 'http://www.google.com/';          # variable to store the contents of the    # downloaded web page    my $downloadedPage = system "curl $url";          # returning the contents using the variable    return $downloadedPage;}  # displaying a user friendly messagesay "the contents of the web page : ";  # calling the sub routinegetWebPage();
Output:
the contents of the downloaded web page :
<raw HTML web page>
LWP::Simple is a module in Perl which provides a get() that takes the URL as a parameter and returns the body of the document. It returns undef if the requested URL cannot be processed by the server.
Perl
#!usr/bin/perl  # using the strict pragmause strict;  # using the warnings pragma to# generate warnings in case of # erroneous codesuse warnings;  # specifying the Perl versionuse 5.010;  # calling the LWP::Simple moduleuse LWP::Simple;  # declaring the sub routinesub getWebPage {      # variable to store the URL     my $url = 'http://www.google.com';          # passing the URL to the get function    # of LWP::Simple module    my $downloadedPage = get $url;          # printing the contents of the web page    say $downloadedPage;}  # displaying a user friendly messagesay 'the contents of the web page are : ';  #calling the sub routinegetWebPage();
Output:
the contents of the downloaded web page :
<raw HTML web page>
HTTP::Tiny is a simple HTTP/1.1 client which implies it is used to get, put, delete, head (basic HTTP actions). It is used for performing simple requests without the overhead of a large framework. First, an HTTP variable is instantiated using the new operator. Next, we get the code for the request by passing the URL in the get method. On successful code, we get the length and the content of the web page at the address of the specified URL. In the case of an unsuccessful code, we display the appropriate message and mention the reasons for the failure of connection.
Perl
#!usr/bin/perl  # using the warnings pragma to# generate warnings in case of # erroneous codeuse warnings;  # specifying the Perl versionuse 5.010;  # calling the HTTP::Tiny moduleuse HTTP::Tiny;  # declaring the sub routinesub getWebPage{      # variable to store the URL    my $url = 'http://www.google.com/';          # instantiating the HTTP variable    my $httpVariable = HTTP::Tiny->new;          # storing the response using the get    # method    my $response = $httpVariable->get($url);          # checking if the code returned successful    if ($response -> {success}){              # specifying the length of the        # web page content using the         # length keyword        say 'the length of the web page : ';        my $length = length $response->{content};        say $length;                  # displaying the contents of the webpage        say 'the contents of the web page are : ';        my $downloadedPage = $response->{content};        say $downloadedPage;    }          # logic for when the code is    # unsuccessful    else{              # displating the reason for failed        # request        say "Failed to establish connection :         $response->{status}.$response->{reasons}";    }}  # calling the sub routinegetWebPage();
Output:
the length of the web page : 
15175
the contents of the web page are :
<html code of the web page>
The approach for the download of multiple web pages using HTTP::Tiny is the same as mentioned above. The only modification is that here the URL of all the web pages are stored in an array and we loop through the array displaying the contents of each web page.
Perl
#!usr/bin/perl  # using the warnings pragma# to generate warnings for# erroneous codeuse warnings;  # specifying the Perl versionuse 5.010;  # calling the HTTP::Tiny moduleuse HTTP::Tiny;  # declaring the sub routinesub getWebPages{      # instantiating the HTTP client    my $httpVariable = HTTP::Tiny->new;          # array of URLs    my @urls = ('http://www.google.com/',    'https://www.geeksforgeeks.org/'    );          # start of foreach loop to    # loop through the array of URLs    foreach my $singleURL (@urls){              # displaying user friendly message        say 'downloading web page...';                  # variable to store the response        my $response = $httpVariable->        get($singleURL);                  # logic for successful connection        if ($response->{success}){            say $singleURL.            " downloaded successfully";                          # displaying the length of            # the web page            # the contents can be displayed            # similarly            say "Length : length             $response->{content}";        }                  # logic for unsuccessful connection        else{            say $singleURL.            " could not be downloaded";                          # displaying the reason for            # unsuccessful connection            say "$response->{status}            $response->{reasons}";        }    }}  # calling the sub routinegetWebPages();
Output:
downloading web page...
downloaded successfully
Length : 15175
<html content of the landing page of google>
downloading web page...
downloaded successfully
Length : <Length of the landing page of GFG>
<html content of the landing page of GFG>
In this section, we will see two approaches to download images using Perl scripts. In order to get the URL of these images, we first right-click on them. Next, we click on Copy Image Address from the drop-down and paste this as the URL for the image.
In this approach, we use LWP::Simple module and get the HTTP code using getstore function. In this function, we have to specify the URL of the image to be downloaded and the location to store the downloaded image. Next, we check if the code is successful or not and display the corresponding message to the user.
Perl
#!usr/bin/perl  # using the strict pragmause strict;  # using the warnings pragma# to generate warnings for# erroneous codeuse warnings;  # specifying the Perl versionuse 5.010;  # calling the moduleuse LWP::Simple;  # declaring the sub routinesub getImage {      # displaying a user friendly message    say "Downloading ... ";          # variable to store the status code    # first parameter is the URL of the image    # second parameter is the location    # of the downloaded image    my $statusCode = getstore    ("https://www.geeksforgeeks.org/wp-content/uploads/gfg_200X200-1.png",    "downloaded_image.png");          # checking for successful    # connection    if ($statusCode == 200) {        say "Image successfully downloaded.";    }    else {        say "Image download failed.";    }}  # calling the sub routinegetImage();
Output:
Downloading...
Image successfully downloaded.
(the downloaded image will be saved at the specified location
with the given name. If no location is specified then the image
would be saved in the current working directory.
Image::Grab is a simple module meant for downloading the images specified by their URLs. It works with images that might be hidden by some method too. In this approach, we use the Image::Grab module and after instantiating it, we pass the URL. Next, we call the grab method and save the downloaded image to disk.
Perl
#!usr/bin/perl  # using the strict pragmause strict;  # using the warnings pragma to# generate warnings for erroneous# codeuse warnings;  # specifying the Perl versionuse 5.010;  # calling the Image::Grab moduleuse Image::Grab;  # instantiating the module# and storing it in a variablemy $instantiatedImage = new Image::Grab;  # declaring the sub routinesub getImage {      # specifying the URL    $instantiatedImage->url    ('https://www.geeksforgeeks.org/wp-content/uploads/gfg_200X200-1.png');          # calling grab to grab the image    $instantiatedImage->grab;          # creating a file to store    # the downloaded image    open(DOWNLOADEDIMAGE, '>downloaded_image1.png') ||                         die'downloaded_image1.png: $!';          # for MSDOS only    binmode DOWNLOADEDIMAGE;          # saving the image in the created    # file    print DOWNLOADEDIMAGE $instantiatedImage->image;          # closing the file    close instantiatedImage;}  # calling the sub routinegetImage();
Output:
The image is stored with the specified file name.
Downloaded Image:
surinderdawra388
Picked
Perl
Perl
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Perl Tutorial - Learn Perl With Examples
Perl | Inheritance in OOPs
Perl | Basic Syntax of a Perl Program
Perl | Opening and Reading a File
Perl | ne operator
Perl | Data Types
Perl | Multidimensional Hashes
Perl | Hashes
Perl | defined() Function
Perl | Scope of Variables