List of articles by Eric Lippert
2023-02-28 perl Mojo::UserAgent Web Scrape IO::Socket::SSL Mozilla::CA Eric LippertEric Lippert is very interesting developer and I greatly enjoyed reading his articles, previously at Microsoft, lately at his blog. The articles are advanced and usually form long series. I have no chance to keep up with them as they are written, so I wanted to build a list of the articles and proceed with reading when I have a time.
So I put together quick perl script to build a markdown list of all articles
use utf8::all;
use Mojo::UserAgent;
my $ua = Mojo::UserAgent->new(max_redirects => 5);
my $base_url = 'https://ericlippert.com';
for my $month ($ua->get($base_url)->res->dom->find('#archives-2 a')->each) {
print "\n## [", $month->text, "](", $month->{href}, ")\n";
for my $article ($ua->get($month->{href})->res->dom->find('article')->each) {
my $title = $article->find('.entry-title > a:nth-child(1)')->first;
my @tags = $article->find('span.tag-links > a')->each;
print " - [ ] [", $title->text, "](", $title->{href}, ")\n";
}
}
It puts together a list of months along with checklist of links like this
## [February 2023](https://ericlippert.com/2023/02/)
- [ ] [Bean Machine Retrospective, part 8](https://ericlippert.com/2023/02/23/bean-machine-retrospective-part-8/)
- [ ] [Bean Machine Retrospective, part 7](https://ericlippert.com/2023/02/08/bean-machine-retrospective-part-7/)
It makes it is easy to check things I worked through and it is also easy to generate file again time from time and add new articles using a diff.
Originally I had trouble with making it work, getting SSL/TLS error
SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
After some digging around internet, I found the description of how it works in IO::Socket::SSL, where it mentioned that Mozilla::CA is used to provide Mozilla’s certificates to openssl modules. After updating of the module with
cpanm Mozilla::CA
everything started working.