• Why WWW::Mechanize::Chrome?

• What is WWW::Mechanize::Chrome?

• Development of WWW::Mechanize::Chrome

• Applications

• Max Maischein

• DZ BANK Frankfurt

• Deutsche Zentralgenossenschaftsbank

• Data Solutions

• If I can do it manually

• ... the computer can repeat it

• ... correctly every time

• Perl (well, duh)

• WWW::Mechanize

• WWW::Mechanize::Shell (GPW 2002)

• WWW::Mechanize::Firefox (YAPC::E 2010)

• WWW::Mechanize::PhantomJS (YAPC::E 2014)

• WWW::Mechanize::Chrome (today)

• (also Chromium, v59+)

• Web applications are still cool

• Web Service Workers are another layer

• PhantomJS has stopped development

• Mozilla+Firefox fight extensions

• Freeze your prerequisites

• Disable auto-update

• Local regression tests

• Travis CI

• Recognized platform

• Compatible platform

• Interactive platform

• WWW::Mechanize::Chrome

• Chrome is a browser, with my cookies

• Chrome wants a UI window

• --headless

• Chrome

• Chrome

• Chrome DevTools

• Chrome

• Chrome DevTools

• WebSocket

• Chrome

• Chrome DevTools

• WebSocket

• AnyEvent or Mojolicious

• Chrome

• Chrome DevTools

• WebSocket

• AnyEvent or Mojolicious

• WWW::Mechanize::Chrome

• Chrome

• Chrome DevTools

• WebSocket

• AnyEvent or Mojolicious

• WWW::Mechanize::Chrome

• My program

• an extended API

• of WWW::Mechanize

• using Chrome as backend

 1:  my $mech = WWW::Mechanize::Chrome->new();
 2:  $mech->get('http://act.yapc.eu/lpw2017/');
 3:  $mech->content_as_png();

• Normal WWW::Mechanize API

• Javascript

• CSS Selectors (via HTML::Selector::XPath)

• XPath Selectors

• Javascript error messages

• Web site automation

• Integrated JS Unit Tests

• Sniffing Websockets

• Android Chrome remote control

Automate Chrome

01-open-local.pl

 1:  my $mech = WWW::Mechanize::Chrome->new();
 2:  $mech->get_local('file.html');

Webseite Test for usability

02-dump-links.pl

 1:  my $mech = WWW::Mechanize::Chrome->new();
 2:  $mech->get_local('links.html');
 3:
 4:  print $_->get_attribute('href'),
 5:        "\n\t-> ",
 6:        $_->get_attribute('innerHTML'), "\n"
 7:    for $mech->selector('a.download');

Run Javascript

03-javascript.pl

 1:  // Javascript
 2:      
 3:      
 4:      
 5:      
 6:      
 7:      " ".join(["Just","another","Perl","Hacker"]);

Run Javascript

03-javascript.pl

 1:  # Perl
 2:  my $mech = WWW::Mechanize::Chrome->new(
 3:      headless => 1,
 4:  );  
 5:  
 6:  print( ($mech->eval_in_page(<<'JS'))[0]);
 7:      " ".join(["Just","another","Perl","Hacker"]);
 8:  JS

• Google Keep Clone

• Javascript+Perl

• Service worker

• Tests

05-screenshot-online.pl

 1:  my $mech = WWW::Mechanize::Chrome->new();
 2:  my $url= 'https://corion.net/notes.psgi';
 3:  print "Loading $url\n";
 4:  $mech->get($url);
 5:  my $page_png = $mech->content_as_png();

06-create-note.pl

 1:  $mech->get($url);
 2:
 3:  $mech->sleep( 5 );
 4:  # Create note
 5:  $mech->eval_in_page(<<'JS', $name);
 6:  ...

06-create-note.pl

 1:  $mech->get($url);
 2:
 3:  $mech->sleep( 5 );
 4:  # Create note
 5:  $mech->eval_in_page(<<'JS', $name);
 6:      var item = {
 7:          title : "Hello London",
 8:          text  : "Created with WWW::Mechanize::Chrome",
 9:          ...
10:      };
11:      saveItem( item );
12:  JS
13:  sleep 1;

PDF output is only available in headless Chrome

07-screenshot-pdf.pl

 1:  my $mech = WWW::Mechanize::Chrome->new(
 2:      headless => 1,
 3:  );
 4:  my $url= 'http://localhost:5000';
 5:  print "Loading $url\n";
 6:  $mech->get($url);
 7:
 8:  $mech->render_content(
 9:      format => 'pdf',
10:      filename => 'screen.pdf'
11:  );

• Alerts (window.alert())

• $mech->on_dialog(...)

• $mech->handle_dialog( 1 );

• Browser Console

• Chrome / Chromium

• WWW::Mechanize::Chrome

• AnyEvent or Mojolicious

• API Implementation (->post() , ...)

• Documentation

• ->post()

NDD

NDD

Need-driven Development

• Simple things first

• ->post() half implemented

• So far no need

• HTTP::Cookies::Chrome for cookie management

API for

• Browser windows (open, close, popup)

• Downloads

• Event API? Callback API?

• Documentation for the API

• WWW::Mechanize::Chrome

• Documentation for FAQs

• WWW::Mechanize::Chrome::Examples

• Rewrite the ::Firefox documentation

• WWW::Mechanize::Chrome::Examples

 1:                 Chrome     PhantomJS      Firefox

 1:                 Chrome     PhantomJS      Firefox
 2:  
 3:  Display        Optional   No             Yes

 1:                 Chrome     PhantomJS      Firefox
 2:  
 3:  Display        Optional   No             Yes
 4:
 5:  Cookies
 6:    persistent   Yes        No             Yes

 1:                 Chrome     PhantomJS      Firefox
 2:  
 3:  Display        Optional   No             Yes
 4:
 5:  Cookies
 6:    persistent   Yes        No             Yes
 7:  Custom
 8:    certificates Ignore     Easy           Hard

 1:                 Chrome     PhantomJS      Firefox
 2:  
 3:  Display        Optional   No             Yes
 4:
 5:  Cookies
 6:    persistent   Yes        No             Yes
 7:  Custom
 8:    certificates Ignore     Easy           Hard
 9:  Dialogs        Yes        Possible       Hard

 1:                 Chrome     PhantomJS      Firefox
 2:  
 3:  Display        Optional   No             Yes
 4:
 5:  Cookies
 6:    persistent   Yes        No             Yes
 7:  Custom
 8:    certificates Ignore     Easy           Hard
 9:  Dialogs        Yes        Possible       Hard
10:  Downloads      No         Yes?           Yes

• Freeze your prerequisites

• Disable auto-update

• Use whatever browser suits your profile

The Good

• Testsuite of WWW::Mechanize::Firefox and ::PhantomJS

• API of WWW::Mechanize

• Experience with ::Firefox

• 32bit App, 64bit Perl -> TCP!

• Future ideal instead of callbacks

• WebSocket trivial using AnyEvent or Mojolicious

• API much more pleasant than Selenium

The Good, the Bad

• Chrome / Chromium is a moving target

• No PDF-support despite documented

• Can't set the Referer header since Chrome v64

The Good, the Bad, the Ugly

• API coverage through tests

• Fine differences between ::Firefox , ::PhantomJS and ::Chrome

• Freeze your prerequisites

• Disable auto-update

The code is on CPAN as

WWW::Mechanize::Chrome::Examples

Questions?

Questions?

Slides are online:

https://corion.net/talks/

WWW::Mechanize::Chrome on CPAN

https://github.com/corion/www-mechanize-chrome on Github

Thank you to the sponsors, without which the London Perl Workshop would not be possible:

• Firefox --headless

• just now Windows

• only Selenium (PhantomJS)

• Firefox hates extensions and XUL

• Screencast mode

• Automatic replay on HTTP errors

• Javascript::SpiderMonkey ( Mike Schili, Thomas Busch on CPAN )

• Javascript::Duktape ( Mahmoud A. Mehyar on CPAN )

• Installable via CPAN, no header files needed

• Windows needs patch

• Javascript::Engine ( Father Chrysostomos/SPROUT on CPAN )

• Pure Perl, slooooow

• JavaScript::Any ( PLICASE on CPAN )

Questions?

Fragen?

Slides are online:

https://corion.net/talks/

WWW::Mechanize::Chrome on CPAN

https://github.com/corion/www-mechanize-chrome on Github