WWW::Mechanize with Perl

I’ve been doing web automation for years at my company. It’s always sort of a pain in the ass, nothing’s all that elegant, lots of klugey crap. I was playing with Perl this weekend, trying to download 2200 pages. Even though I’ve never really used Perl before, I wrote a script that logged in, and downloaded 2200 pages in about an hour. Even knowing .NET like the back of my hand this would have taken about the same amount of time. Thanks CPAN!

What’s nice about WWW::Mechanize, over the the .NET/VB6 methods we use (which relies on constructing post/get strings and managing cookies) is it allows you to set form fields and “click” form submit. This means you don’t need to hunt down the 15 zillion hidden fields on the form or handle the redirect in the javascript submit or any of that other nonsense. Much easier.

Code is here if you’re interested (simple script to down/steal all recipes from the SouthBitch diet).