shawnhcorey was gonna look at my code to figure how to grab (hash)tags.

Mastodon::Client does implement the search part of the Mastodon API. Kinda. I haven't dug too deeply into that. I don't think the tags api is directly implemented but the search method can stand in.

This post is not about that, it's about how to do pagination.

I'll do this in chunks.

This is just loading modules I'll be using. I am assuming you are familiar with Perl and what is going on here. Some are needed to make Mastodon::Client happy.

#!/usr/bin/env perl

use Mastodon::Client;

use v5.38;

use Mastodon::Client;
use YAML qw(LoadFile);
use JSON::MaybeXS;
use Type::Params qw( compile validate );
use HTTP::Response;

Now the subroutine to get bookmarks.

Covered in the previous blog post about why

sub Mastodon::Client::get_bookmarks {
  my $self = shift;
  my ($params) = @_;

  my $endpoint = q(/bookmarks);

  return $self->get($endpoint, $params);
}

Now for the big check, most of which doesn't relate to pagination. I've removed a bunch of code relating to parsing bookmark info. Look in the Mastodon API about bookmarks to see the data that comes back in the response.

sub parse_bookmarks ($bookmarks, $client) {
  my $links = $client->latest_response->{_headers}{link};

  my $pagination;

  RAW : for my $raw ( split(/,/, $links ) ) {
    my ($url_raw, $rel) = split(/;/, $raw);
    $url_raw =~ s/[<>]//g;
    my ($direction) = $rel =~ m/rel="(\w+)"/;
    my ($url, $params_returned) = split(/\?/, $url_raw);

    say qq(  direction : $direction);
    PARAM : for my $param ( split/&/, $params_returned) {
      my ($name, $value) = split(/=/, $param);
      $pagination->{$direction}{$name} = $value;
      say qq(    name      : $name);
      say qq(    value     : $value);
    } # PARAM
  } # RAW

  return($pagination);
};

Credentials and such are stored in a config file, YAML is my current format of choice.

You will need to read the API docs to create an application

My YAML file looks like:

instance: INSTANCE_URL
name: REGISTERED_APP_NAME
user_id_number: #####
key: KEY
secret: SECRET
token: TOKEN

Now just let the program know where to file the file and then load it.

my $config_file = q(./gmb-masto-reader.yml);
my $config = LoadFile($config_file);

Create the client and give it the parameters it needs.

my $client = Mastodon::Client->new(
  instance        => $config->{instance},
  name            => $config->{name},
  client_id       => $config->{key},
  client_secret   => $config->{secret},
  access_token    => $config->{token},
);

Get some bookmarks.

my $bookmarks = $client->get_bookmarks(
  {
    limit => 40,
  }
);

Do a call to parsing the bookmarks or whatever to kick off the stepping through pagination.

With max_id and min_id you get older things with max_id and new things with min_id (I think, max_id gives me what I want.)

my ($pagination) = parse_bookmarks($bookmarks, $client);

say qq(  pagination_next_max_id : $pagination->{next}{max_id});

Now you can loop until the well runs dry or do a for loop and just do a couple of pages.

while ( $pagination->{next}{max_id} > 0  ) {
  $bookmarks = $client->get_bookmarks( $pagination->{next}, $client );
  ($pagination) = parse_bookmarks($bookmarks, $client);

  say q();
  sleep(1);
}

And that's basically it.

Here's the full thing in its ugly glory, including stuff I took out.

#!/usr/bin/env perl

use Mastodon::Client;

use v5.38;

use Mastodon::Client;
use YAML qw(LoadFile);
use Data::Printer;
use JSON::MaybeXS;
use Type::Params qw( compile validate );
use HTTP::Response;

sub Mastodon::Client::get_bookmarks {
  my $self = shift;
  my ($params) = @_;

  my $endpoint = q(/bookmarks);

  return $self->get($endpoint, $params);
}
###
sub parse_bookmarks ($bookmarks, $client) {
  say qq(bookmarked :  $bookmarks->[0]->{bookmarked});
  my $links = $client->latest_response->{_headers}{link};

  my $data = decode_json($client->latest_response->{_content});


  BOOKMARK_DATA : for my $bookmark_data ( $data->@* ) {
    my $id         = $bookmark_data->{id};
    my $created_at = $bookmark_data->{'created_at'};
    my $url        = $bookmark_data->{url};
    my $content    = $bookmark_data->{content};

    say qq( id : $id);
    say qq( url : $url);
    say qq( created_at : $created_at);
    say qq( content : $content);
  } # BOOKMARK_DATA

  say qq(\n\nlinks : $links);

  my $pagination;

  RAW : for my $raw ( split(/,/, $links ) ) {
    my ($url_raw, $rel) = split(/;/, $raw);
    $url_raw =~ s/[<>]//g;
    my ($direction) = $rel =~ m/rel="(\w+)"/;
    my ($url, $params_returned) = split(/\?/, $url_raw);

    say qq(  direction : $direction);
    PARAM : for my $param ( split/&/, $params_returned) {
      my ($name, $value) = split(/=/, $param);
      $pagination->{$direction}{$name} = $value;
      say qq(    name      : $name);
      say qq(    value     : $value);
    } # PARAM
  } # RAW

  return($pagination);
};

my $config_file = q(./gmb-masto-reader.yml);
my $config = LoadFile($config_file);

my $client = Mastodon::Client->new(
  instance        => $config->{instance},
  name            => $config->{name},
  client_id       => $config->{key},
  client_secret   => $config->{secret},
  access_token    => $config->{token},
);
my $bookmarks = $client->get_bookmarks(
  {
    limit => 40,
  }
  );

my ($pagination) = parse_bookmarks($bookmarks, $client);

say qq(  pagination_next_max_id : $pagination->{next}{max_id});

while ( $pagination->{next}{max_id} > 0  ) {
  $bookmarks = $client->get_bookmarks( $pagination->{next}, $client );
  ($pagination) = parse_bookmarks($bookmarks, $client);

  say q();
  sleep(1);
}