Recipe 20.15 Fetching Password-Protected Pages

20.15.1 Problem

You want to use LWP to fetch web pages or submit forms, but the web server requires authentication.

20.15.2 Solution

Set the username and password for a particular realm with the user agent's credentials method:

$ua->credentials('http://www.perlcabal.com/cabal_only/',
                 'Secret Perl Cabal Files',
                 'username' => 'password');

20.15.3 Discussion

To access pages protected by basic authentication, a browser must supply the username and password for the realm of the authentication. The realm is just a string that identifies which username and password the user must supply. The credentials method tells the user agent to send the username and password for a particular realm.

A somewhat kludgey solution is to specify URLs with the username and password in them:

http://user:password@www.example.com/private/pages/

This is kludgey because links within the returned document do not have your username and password encoded in them. Solutions that rely entirely on URL-encoded usernames and passwords often quickly degenerate into code that wishes it had used credentials to begin with.

20.15.4 See Also

The documentation for the CPAN module LWP::UserAgent