List of articles by Eric Lippert

2023-02-28 perl Mojo::UserAgent Web Scrape IO::Socket::SSL Mozilla::CA Eric Lippert

Eric Lippert is very interesting developer and I greatly enjoyed reading his articles, previously at Microsoft, lately at his blog. The articles are advanced and usually form long series. I have no chance to keep up with them as they are written, so I wanted to build a list of the articles and proceed with reading when I have a time.

So I put together quick perl script to build a markdown list of all articles

use utf8::all;
use Mojo::UserAgent;

my $ua = Mojo::UserAgent->new(max_redirects => 5);

my $base_url = 'https://ericlippert.com';
for my $month ($ua->get($base_url)->res->dom->find('#archives-2 a')->each) {
    print "\n## [", $month->text, "](", $month->{href}, ")\n";
    for my $article ($ua->get($month->{href})->res->dom->find('article')->each) {
        my $title = $article->find('.entry-title > a:nth-child(1)')->first;
        my @tags  = $article->find('span.tag-links > a')->each;
        print " - [ ] [", $title->text, "](", $title->{href}, ")\n";
    }
}

It puts together a list of months along with checklist of links like this

## [February 2023](https://ericlippert.com/2023/02/)
 - [ ] [Bean Machine Retrospective, part 8](https://ericlippert.com/2023/02/23/bean-machine-retrospective-part-8/)
 - [ ] [Bean Machine Retrospective, part 7](https://ericlippert.com/2023/02/08/bean-machine-retrospective-part-7/)

It makes it is easy to check things I worked through and it is also easy to generate file again time from time and add new articles using a diff.

Originally I had trouble with making it work, getting SSL/TLS error

SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed

After some digging around internet, I found the description of how it works in IO::Socket::SSL, where it mentioned that Mozilla::CA is used to provide Mozilla’s certificates to openssl modules. After updating of the module with

cpanm Mozilla::CA

everything started working.