constant IterationEnd
role Iterator { }

A Iterator is an object that can generate or provide elements of a sequence. Users usually don't have to care about iterators, their usage is hidden behind iteration APIs such as for @list { }, map, grep, head, tail, skip and list indexing with .[$idx].

The main API is the pull-one method, which either returns the next value, or the sentinel value IterationEnd if no more elements are available. Each class implementing Iterator must provide a pull-one method. All other non-optional Iterator API methods are implemented in terms of pull-one, but can also be overridden by consuming classes for performance or other reasons. There are also optional Iterator API methods that will only be called if they are implemented by the consuming class: these are not implemented by the Iterator role.

IterationEnd

Iterators only allow one iteration over the entire sequence. It's forbidden to make attempts to fetch more data, once IterationEnd has been generated, and behavior for doing so is undefined. For example, the following Seq will not cause the die to be called under normal use, because pull-one will never be called after it returns IterationEnd:

class SkippingArray is Array {
    # skip all undefined values while iterating 
    method iterator {
        class :: does Iterator {
            has $.index is rw = 0;
            has $.array is required;
            method pull-one {
                $.index++ while !$.array.AT-POS($.index).defined && $.array.elems > $.index;
                $.array.elems > $.index ?? $.array.AT-POS($.index++) !! IterationEnd
            }
        }.new(array => self)
    }
}
 
my @a := SkippingArray.new;
 
@a.append: 1, Any, 3, Int, 5, Mu, 7;
 
for @a -> $a, $b {
    say [$a, $b];
}
 
# OUTPUT: «[1 3]␤[5 7]␤» 

The only valid use of the sentinel value IterationEnd in a program is identity comparison (using =:=) with the result of a method in the iterator API. Any other behavior is undefined and implementation dependent. For instance, using it as part of a list that's going to be iterated over might result in the loop ending, or not.

.say for ["foo",IterationEnd, "baz"]; # OUTPUT: «foo␤» 
say ["foo",IterationEnd, "baz"].map: "«" ~ * ~ "»";
# OUTPUT: «(«foo» «IterationEnd» «baz»)␤» 

Please bear in mind that IterationEnd is a constant, so if you are going to compare it against the value of a variable, this variable will have to be bound, not assigned. Comparing directly to the output of pull-one will work.

my $it = (1,2).iterator;
$it.pull-one for ^2;
say $it.pull-one =:= IterationEnd; # OUTPUT: «True␤» 

However, if we use a variable we and we assign it, the result will be incorrect:

my $it = (1,2).iterator;
$it.pull-one for ^2;
my $is-it-the-end = $it.pull-one;
say $is-it-the-end =:= IterationEnd; # OUTPUT: «False␤» 

So we'll have to bind the variable to make it work:

my $is-it-the-end := $it.pull-one;
say $is-it-the-end =:= IterationEnd; # OUTPUT: «True␤» 

Methods

method pull-one

method pull-one(Iterator:D: --> Mu)

This method stub ensures that classes implementing the Iterator role provide a method named pull-one.

The pull-one method is supposed to produce and return the next value if possible, or return the sentinel value IterationEnd if no more values could be produced.

my $i = (1 .. 3).iterator;
say $i.pull-one;       # OUTPUT: «1␤» 
say $i.pull-one;       # OUTPUT: «2␤» 
say $i.pull-one;       # OUTPUT: «3␤» 
say $i.pull-one.raku;  # OUTPUT: «IterationEnd␤»

As a more illustrative example of its use, here is a count down iterator along with a simplistic subroutine re-implementation of the for loop.

# works the same as (10 ... 1, 'lift off') 
class CountDown does Iterator {
    has Int:D $!current = 10;
 
    method pull-one ( --> Mu ) {
        my $result = $!current--;
        if $result ==  0 { return 'lift off' }
        if $result == -1 { return IterationEnd }
 
        # calling .pull-one again after it returns IterationEnd is undefined 
        if $result <= -2 {
            # so for fun we will give them nonsense data 
            return (1..10).pick;
        }
 
        return $result;
    }
}
 
sub for( Iterable:D $sequence, &do --> Nil ) {
    my Iterator:D $iterator = $sequence.iterator;
 
    loop {
        # must bind the result so that =:= works 
        my Mu $pulled := $iterator.pull-one;
 
        # always check the result and make sure that .pull-one 
        # is not called again after it returns IterationEnd 
        if $pulled =:= IterationEnd { last }
 
        do( $pulled );
    }
}
 
for( Seq.new(CountDown.new), &say );  # OUTPUT: «10␤9␤8␤7␤6␤5␤4␤3␤2␤1␤lift off␤»

It would be more idiomatic to use while or until, and a sigilless variable.

until IterationEnd =:= (my \pulled = $iterator.pull-one) {
    do( pulled );
}

method push-exactly

method push-exactly(Iterator:D: $target, int $count --> Mu)

Should produce $count elements, and for each of them, call $target.push($value).

If fewer than $count elements are available from the iterator, it should return the sentinel value IterationEnd. Otherwise it should return $count.

my @array;
say (1 .. ∞).iterator.push-exactly(@array, 3); # OUTPUT: «3␤» 
say @array; # OUTPUT: «[1 2 3]␤»

The Iterator role implements this method in terms of pull-one. In general, this is a method that is not intended to be called directly from the end user who, instead, should implement it in classes that mix the iterator role. For instance, this class implements that role:

class DNA does Iterable does Iterator {
    has $.chain;
    has Int $!index = 0;
 
    method new ($chain where {
                       $chain ~~ /^^ <[ACGT]>+ $$ / and
                       $chain.chars %% 3 } ) {
        self.bless( :$chain );
    }
 
    method iterator( ){ self }
 
    method pull-one( --> Mu){
      if $!index < $.chain.chars {
         my $codon = $.chain.comb.rotor(3)[$!index div 3];
         $!index += 3;
         return $codon;
      } else {
        return IterationEnd;
      }
    }
 
    method push-exactly(Iterator:D: $target, int $count --> Mu) {
        return IterationEnd if $.chain.elems / 3 < $count;
        for ^($count) {
            $target.push: $.chain.comb.rotor(3)[ $_ ];
        }
    }
 
};
 
my $b := DNA.new("AAGCCT");
for $b -> $a, $b, $c { say "Never mind" }; # Does not enter the loop 
my $þor := DNA.new("CAGCGGAAGCCT");
for $þor -> $first, $second {
    say "Coupled codons: $first, $second";
    # OUTPUT: «Coupled codons: C A G, C G G␤Coupled codons: A A G, C C T␤» 
}

This code, which groups DNA chains in triplets (usually called codons) returns those codons when requested in a loop; if too many are requested, like in the first case for $b -> $a, $b, $c, it simply does not enter the loop since push-exactly will return IterationEnd since it is not able to serve the request for exactly 3 codons. In the second case, however, it requests exactly two codons in each iteration of the loop; push-exactly is being called with the number of loop variables as the $count variable.

method push-at-least

method push-at-least(Iterator:D: $target, int $count --> Mu)

Should produce at least $count elements, and for each of them, call $target.push($value).

If fewer than $count elements are available from the iterator, it should return the sentinel value IterationEnd. Otherwise it should return $count.

Iterators with side effects should produce exactly $count elements; iterators without side effects (such as Range iterators) can produce more elements to achieve better performance.

my @array;
say (1 .. ∞).iterator.push-at-least(@array, 10); # OUTPUT: «10␤» 
say @array; # OUTPUT: «[1 2 3 4 5 6 7 8 9 10]␤»

The Iterator role implements this method in terms of pull-one. In general, it is also not intended to be called directly as in the example above. It can be implemented, if unhappy with this default implementation, by those using this role. See the documentation for push-exactly for an example implementation.

method push-all

method push-all(Iterator:D: $target)

Should produce all elements from the iterator and push them to $target.

my @array;
say (1 .. 1000).iterator.push-all(@array); # All 1000 values are pushed

The Iterator role implements this method in terms of push-at-least. As in the case of the other push-* methods, it is mainly intended for developers implementing this role. push-all is called when assigning an object with this role to an array, for instance, like in this example:

class DNA does Iterable does Iterator {
    has $.chain;
    has Int $!index = 0;
 
    method new ($chain where {
                       $chain ~~ /^^ <[ACGT]>+ $$ / and
                       $chain.chars %% 3 } ) {
        self.bless( :$chain );
    }
 
    method iterator( ){ self }
    method pull-one( --> Mu){
      if $!index < $.chain.chars {
         my $codon = $.chain.comb.rotor(3)[$!index div 3];
         $!index += 3;
         return $codon;
      } else {
        return IterationEnd;
      }
    }
 
    method push-all(Iterator:D: $target) {
        for $.chain.comb.rotor(3) -> $codon {
            $target.push: $codon;
        }
    }
 
};
 
my $b := DNA.new("AAGCCT");
my @dna-array = $b;
say @dna-array; # OUTPUT: «[(A A G) (C C T)]␤» 

The push-all method implemented pushes to the target iterator in lists of three amino acid representations; this is called under the covers when we assign $b to @dna-array.

method push-until-lazy

method push-until-lazy(Iterator:D: $target --> Mu)

Should produce values until it considers itself to be lazy, and push them onto $target.

The Iterator role implements this method as a no-op if is-lazy returns a True value, or as a synonym of push-all if not.

This matters mostly for iterators that have other iterators embedded, some of which might be lazy, while others aren't.

method is-deterministic

method is-deterministic(Iterator:D: --> Bool:D)

Should return True for iterators that, given a source, will always produce the values in the same order.

Built-in operations can perform certain optimizations when it is certain that values will always be produced in the same order. Some examples:

say (1..10).iterator.is-deterministic;              # OUTPUT: «True␤» 
say (1..10).roll(5).iterator.is-deterministic;      # OUTPUT: «False␤» 
say %(a => 42, b => 137).iterator.is-deterministic; # OUTPUT: «False␤»

The Iterator role implements this method returning True, indicating an iterator that will always produce values in the same order.

method is-lazy

method is-lazy(Iterator:D: --> Bool:D)

Should return True for iterators that consider themselves lazy, and False otherwise.

Built-in operations that know that they can produce infinitely many values return True here, for example (1..6).roll(*).

say (1 .. 100).iterator.is-lazy; # OUTPUT: «False␤» 
say (1 .. ∞).iterator.is-lazy; # OUTPUT: «True␤»

The Iterator role implements this method returning False, indicating a non-lazy iterator.

method sink-all

method sink-all(Iterator:D: --> IterationEnd)

Should exhaust the iterator purely for the side-effects of producing the values, without actually saving them in any way. Should always return IterationEnd. If there are no side-effects associated with producing a value, then it can be implemented by a consuming class to be a virtual no-op.

say (1 .. 1000).iterator.sink-all;  # OUTPUT: «IterationEnd␤»

The Iterator role implements this method as a loop that calls pull-one until it is exhausted.

method skip-one

method skip-one(Iterator:D: $target --> Mu)

Should skip producing one value. The return value should be truthy if the skip was successful and falsy if there were no values to be skipped:

my $i = <a b>.iterator;
say $i.skip-one; say $i.pull-one; say $i.skip-one
# OUTPUT: «1␤b␤0␤»

The Iterator role implements this method as a call pull-one and returning whether the value obtained was not IterationEnd.

method skip-at-least

method skip-at-least(Iterator:D: $target, int $to-skip --> Mu)

Should skip producing $to-skip values. The return value should be truthy if the skip was successful and falsy if there were not enough values to be skipped:

my $i = <a b c>.iterator;
say $i.skip-at-least(2); say $i.pull-one; say $i.skip-at-least(20);
# OUTPUT: «1␤c␤0␤»

The Iterator role implements this method as a loop calling skip-one and returning whether it returned a truthy value sufficient number of times.

method skip-at-least-pull-one

method skip-at-least-pull-one(Iterator:D: $target, int $to-skip --> Mu)

Should skip producing $to-skip values and if the iterator is still not exhausted, produce and return the next value. Should return IterationEnd if the iterator got exhausted at any point:

my $i = <a b c>.iterator;
say $i.skip-at-least-pull-one(2);
say $i.skip-at-least-pull-one(20) =:= IterationEnd;
# OUTPUT: «c␤True␤»

The Iterator role implements this method as calling skip-at-least and then calling pull-one if it was not exhausted yet.

Predictive iterators

Please see the PredictiveIterator role if your Iterator can know how many values it can still produce without actually producing them.