Indented Here Documents

These were introduced in Perl 5.25.7 (November 2016) but not recognized by this module until its version 0.015 (February 2021). The indentation is parsed as PPIx::QuoteLike::Token::Whitespace objects, provided it is at least one character wide, otherwise it is not represented in the parse. That is to say,

<<~EOD
    How doth the little crocodile
    Improve his shining tail
    EOD

will have the three indentations represented by whitespace objects and each line of the literal represented by its own string object, but

<<~EOD
How doth the little crocodile
Improve his shining tail
EOD

will parse the same as the non-indented version, except for the addition of the token representing the '~'.

PPI is ahead of this module, and recognized indented here documents as of its version 1.246 (May 2019). Unfortunately, as of version 1.270 the indent gets lost in the parse, so a "PPIx::QuoteLike" object initialized from such a PPI::Token::HereDoc will be seen as having an indentation of '' regardless of the actual indentation in the source. I believe this restriction will go away when https://github.com/Perl-Critic/PPI/issues/251 is resolved.

new

my $str = PPIx::QuoteLike->new( $source, %arg );

This static method parses the argument, and returns a new object containing the parse. The $source argument can be either a scalar or an appropriate PPI::Element object.

If the $source argument is a scalar, it is presumed to represent a quote-like literal of some sort, provided it begins like one. Otherwise this method will return nothing. The scalar representation of a here document is a multi-line string whose first line consists of the leading " << " and the start delimiter, and whose subsequent lines consist of the content of the here document and the end delimiter. Indented here documents were not supported by this class until version 0.015.

"PPI" classes that can be handled are PPI::Token::Quote, PPI::Token::QuoteLike::Backtick, PPI::Token::QuoteLike::Command, PPI::Token::QuoteLike::Readline, and PPI::Token::HereDoc. Any other object will cause new() to return nothing.

Additional optional arguments can be passed as name/value pairs. Supported arguments are:

encoding

This is the encoding of the $source. If this is specified as something other than "undef", the $source will be decoded before processing.

If the $source is a "PPI::Element", this encoding is used only if the document that contains the element has neither a byte order mark nor 'use utf8'.

index_locations

This Boolean argument determines whether the locations of the tokens should be computed. It defaults to true if the $source argument is a PPI::Element or if the "location" argument was provided, and false otherwise.

location

This argument is a reference to an array compatible with that returned by the PPI::Element location() method. It defaults to the location of the $source argument if that was a PPI::Element, otherwise no locations will be available.

trace

This Boolean argument causes a trace of the parse to be written to standard out. Setting this to a true value is unsupported in the sense that the author makes no representation as to what will happen if you do it, and reserves the right to make changes to the functionality, or retract it completely, without notice.

All other arguments are unsupported and reserved to the author.

child

my $kid = $str->child( 0 );

This method returns the child element whose index is given as the argument. Children do not include the type(), or the start() or finish() delimiters. Negative indices are valid, and given the usual Perl interpretation.

children

my @kids = $str->children();

This method returns all child elements. Children do not include the type(), or the start() or finish() delimiters.

column_number

This method returns the column number of the first character in the element, or "undef" if that can not be determined.

content

say $str->content();

This method returns the content of the object. If the original argument was a valid Perl string, this should be the same as the originally-parsed string.

delimiters

say $str->delimiters();

This method returns the delimiters of the object, as a string. This will be two characters unless the argument to new() was a here document, missing its end delimiter, or an invalid string. In the latter case the return might be anything.

elements

my @elem = $str->elements();

This method returns all elements of the object. This includes type(), start(), children(), and finish(), in that order.

failures

say $str->failures();

This method returns the number of parse failures found. These are instances where the parser could not figure out what was going on, and should be the same as the number of PPIx::QuoteLike::Token::Unknown objects returned by elements().

find

for ( @{[ $str->find( $criteria ) || [] } ) {
    ...
}

This method finds and returns a reference to an array of all elements that meet the given criteria. If nothing is found, a false value is returned.

The $criteria can be either the name of a PPIx::QuoteLike::Token class, or a code reference. In the latter case, the code is called for each element in elements(), with the element as the only argument. The element is included in the output if the code returns a true value.

finish

say map { $_->content() } $str->finish();

This method returns the finishing elements of the parse. It is actually an array, with the first element being a PPIx::QuoteLike::Token::Delimiter. If the parse is of a here document there will be a second element, which will be a PPIx::QuoteLike::Token::Whitespace containing the trailing new line character.

If called in list context you get the whole array. If called in scalar context you get the element whose index is given in the argument, or element zero if no argument is specified.

handles

say PPIx::QuoteLike->handles( $string ) ?
    "We can handle $string" :
    "We can not handle $string";

This convenience static method returns a true value if this package can be expected to handle the content of $string (be it scalar or object), and a false value otherwise.

indentation

This method returns the indentation string if the object represents an indented here document, or "undef" if it represents anything else, including an unindented here document.

Note that if indented syntax is used but the here document is not in fact indented, this will return '', which evaluates to false.

interpolates

say $str->interpolates() ?
    'The string interpolates' :
    'The string does not interpolate';

This method returns a true value if the parsed string interpolates, and a false value if it does not. This does not indicate whether any interpolation actually takes place, only whether the string is double-quotish or single-quotish.

line_number

This method returns the line number of the first character in the element, or "undef" if that can not be determined.

location

This method returns a reference to an array describing the position of the string, or "undef" if the location is unavailable.

The array is compatible with the corresponding PPI::Element method.

logical_filename

This method returns the logical file name (taking "#line" directives into account) of the file containing first character in the element, or "undef" if that can not be determined.

logical_line_number

This method returns the logical line number (taking "#line" directives into account) of the first character in the element, or "undef" if that can not be determined.

parent

This method returns nothing, since the invocant is only used at the top of the object hierarchy.

perl_version_introduced

This method returns the maximum value of "perl_version_introduced" returned by any of its elements. In other words, it returns the minimum version of Perl under which this quote-like object is valid. If there are no elements, 5.000 is returned, since that is the minimum value of Perl supported by this package.

perl_version_removed

This method returns the minimum defined value of "perl_version_removed" returned by any of the quote-like object's elements. In other words, it returns the lowest version of Perl in which this object is "not" valid. If there are no elements, or if no element has a defined "perl_version_removed", "undef" is returned.

schild

my $skid = $str->schild( 0 );

This method returns the significant child elements whose index is given by the argument. Negative indices are interpreted in the usual way.

schildren

my @skids = $str->schildren();

This method returns the significant children.

source

my $source = $str->source();

This method returns the $source argument to new(), whatever it was.

start

say map { $_->content() } $str->start();

This method returns the starting elements of the parse. It is actually an array, with the first element being a PPIx::QuoteLike::Token::Delimiter. If the parse is of a here document there will be a second element, which will be a PPIx::QuoteLike::Token::Whitespace containing the trailing new line character.

If called in list context you get the whole array. If called in scalar context you get the element whose index is given in the argument, or element zero if no argument is specified.

statement

This method returns the PPI::Statement that contains this string, or nothing if the statement can not be determined.

In general this method will return something only under the following conditions:

• The string is contained in a PPIx::QuoteLike object;
• That object was initialized from a PPI::Element;
• The PPI::Element is contained in a statement.

top

This method returns the top of the hierarchy -- in this case, the invocant.

type

my $type = $str->type();

This method returns the type object. This will be a PPIx::QuoteLike::Token::Structure if the parse was successful; otherwise it might be "undef". Its contents will be everything up to the start delimiter, and will typically be 'q', 'qq', 'qx', '<<' (for here documents), or '' (for quoted strings).

The type data are actually an array. If the second element is present it will be the white space (if any) separating the actual type from the value. If called in list context you get the whole array. If called in scalar context you get the element whose index is given in the argument, or element zero if no argument is specified.

variables

say "Interpolates $_" for $str->variables();

NOTE that this method is discouraged, and may well be deprecated and removed. I have two problems with it. The first is that it returns variable names rather than PPI::Element objects, leaving you no idea how the variables are used. The second is that it does not properly handle things like "${^CAPTURE[0]}", and it seems infeasible to make it do so. It was originally written for the benefit of Perl::Critic::Policy::Variables::ProhibitUnusedVarsStricter, but has proven inadequate to that policy's needs.

This convenience method returns all interpolated variables. Each is returned only once, and they are returned in no particular order. If the object does not represent a string that interpolates, nothing is returned.

visual_column_number

This method returns the visual column number (taking tabs into account) of the first character in the element, or "undef" if that can not be determined.

Changes in Syntax

Sometimes the introduction of new syntax changes the way a string is parsed. For example, the "\F" (fold case) case control was introduced in Perl 5.15.8. But it did not represent a syntax error prior to that version of Perl, it was simply parsed as "F". So

$ perl -le 'print "Foo\FBar"'

prints "FooFBar" under Perl 5.14.4, but "Foobar" under 5.16.0. "PPIx::QuoteLike" generally assumes the more modern parse in cases like this.

Static Parsing

It is well known that Perl can not be statically parsed. That is, you can not completely parse a piece of Perl code without executing that same code.

Nevertheless, this class is trying to statically parse quote-like things. I do not have any examples of where the parse of a quote-like thing would change based on what is interpolated, but neither can I rule it out. Caveat user.

PPI Restrictions

As of version 0.015 of this module, the only known instance of this is the handling of indented here documents, as discussed above under Indented Here Documents.

Non-Standard Syntax

There are modules out there that alter the syntax of Perl. If the syntax of a quote-like string is altered, this module has no way to understand that it has been altered, much less to adapt to the alteration. The following modules are known to cause problems:

Acme::PerlML, which renders Perl as XML.

"Data::PostfixDeref", which causes Perl to interpret suffixed empty brackets as dereferencing the thing they suffix. This module by Ben Morrow ("BMORROW") appears to have been retracted.

Filter::Trigraph, which recognizes ANSI C trigraphs, allowing Perl to be written in the ISO 646 character set.

Perl6::Pugs. Enough said.