.\" -*- mode: troff; coding: utf-8 -*- .\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. .ie n \{\ . ds C` "" . ds C' "" 'br\} .el\{\ . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "Unicode::Stringprep 3pm" .TH Unicode::Stringprep 3pm 2024-09-01 "perl v5.40.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH NAME Unicode::Stringprep \- Preparation of Internationalized Strings (RFC\ 3454) .SH SYNOPSIS .IX Header "SYNOPSIS" .Vb 3 \& use Unicode::Stringprep; \& use Unicode::Stringprep::Mapping; \& use Unicode::Stringprep::Prohibited; \& \& my $prepper = Unicode::Stringprep\->new( \& 3.2, \& [ { 32 => \*(Aq\*(Aq}, ], \& \*(AqKC\*(Aq, \& [ @Unicode::Stringprep::Prohibited::C12, @Unicode::Stringprep::Prohibited::C22, \& @Unicode::Stringprep::Prohibited::C3, @Unicode::Stringprep::Prohibited::C4, \& @Unicode::Stringprep::Prohibited::C5, @Unicode::Stringprep::Prohibited::C6, \& @Unicode::Stringprep::Prohibited::C7, @Unicode::Stringprep::Prohibited::C8, \& @Unicode::Stringprep::Prohibited::C9 ], \& 1, 0 ); \& $output = $prepper\->($input) .Ve .SH DESCRIPTION .IX Header "DESCRIPTION" This module implements the \fIstringprep\fR framework for preparing Unicode text strings in order to increase the likelihood that string input and string comparison work in ways that make sense for typical users throughout the world. The \fIstringprep\fR protocol is useful for protocol identifier values, company and personal names, internationalized domain names, and other text strings. .PP The \fIstringprep\fR framework does not specify how protocols should prepare text strings. Protocols must create profiles of stringprep in order to fully specify the processing options. .SH FUNCTIONS .IX Header "FUNCTIONS" This module provides a single function, \f(CW\*(C`new\*(C'\fR, that creates a perl function implementing a \fIstringprep\fR profile. .PP This module exports nothing. .ie n .IP "\fBnew($unicode_version, \fR\fB$mapping_tables\fR\fB, \fR\fB$unicode_normalization\fR\fB, \fR\fB$prohibited_tables\fR\fB, \fR\fB$bidi_check\fR\fB, \fR\fB$unassigned_check\fR\fB)\fR" 4 .el .IP "\fBnew($unicode_version, \fR\f(CB$mapping_tables\fR\fB, \fR\f(CB$unicode_normalization\fR\fB, \fR\f(CB$prohibited_tables\fR\fB, \fR\f(CB$bidi_check\fR\fB, \fR\f(CB$unassigned_check\fR\fB)\fR" 4 .IX Item "new($unicode_version, $mapping_tables, $unicode_normalization, $prohibited_tables, $bidi_check, $unassigned_check)" Creates a \f(CW\*(C`bless\*(C'\fRed function reference that implements a stringprep profile. .Sp This function takes the following parameters: .RS 4 .ie n .IP $unicode_version 4 .el .IP \f(CW$unicode_version\fR 4 .IX Item "$unicode_version" The Unicode version specified by the stringprep profile. .Sp Currently, this parameter must be \f(CW3.2\fR (numeric). .ie n .IP $mapping_tables 4 .el .IP \f(CW$mapping_tables\fR 4 .IX Item "$mapping_tables" The mapping tables used for stringprep. .Sp The parameter may be a reference to a hash or an array, or \f(CW\*(C`undef\*(C'\fR. A hash must map Unicode codepoints (as integers, e.\ g. \f(CW0x0020\fR for U+0020) to replacement strings (as perl strings). An array may contain pairs of Unicode codepoints and replacement strings as well as references to nested hashes and arrays. .Sp Unicode::Stringprep::Mapping provides the tables from RFC\ 3454, Appendix\ B. .Sp For further information on the mapping step, see RFC\ 3454, section\ 3. .ie n .IP $unicode_normalization 4 .el .IP \f(CW$unicode_normalization\fR 4 .IX Item "$unicode_normalization" The Unicode normalization to be used. .Sp Currently, \f(CW\*(C`undef\*(C'\fR/\f(CW\*(Aq\*(Aq\fR (no normalization) and \f(CW\*(AqKC\*(Aq\fR (compatibility composed) are specified for \fIstringprep\fR. .Sp For further information on the normalization step, see RFC\ 3454, section\ 4. .Sp Normalization form KC will also enable checks for some problem sequences for which the normalization can't be implemented in an interoperable way. .Sp For more information, see "CAVEATS" below. .ie n .IP $prohibited_tables 4 .el .IP \f(CW$prohibited_tables\fR 4 .IX Item "$prohibited_tables" The list of prohibited output characters for stringprep. .Sp The parameter may be a reference to an array, or \f(CW\*(C`undef\*(C'\fR. The array contains \fBpairs\fR of codepoints, which define the \fBstart\fR and \fBend\fR of a Unicode character range (as integers). The end character may be \f(CW\*(C`undef\*(C'\fR, specifying a single-character range. The array may also contain references to nested arrays. .Sp Unicode::Stringprep::Prohibited provides the tables from RFC\ 3454, Appendix\ C. .Sp For further information on the prohibition checking step, see RFC\ 3454, section\ 5. .ie n .IP $bidi_check 4 .el .IP \f(CW$bidi_check\fR 4 .IX Item "$bidi_check" Whether to employ checks for confusing bidirectional text. A boolean value. .Sp For further information on the bidi checking step, see RFC\ 3454, section\ 6. .ie n .IP $unassigned_check 4 .el .IP \f(CW$unassigned_check\fR 4 .IX Item "$unassigned_check" Whether to check for and prohibit unassigned characters. A boolean value. .Sp The check must be used when creating \fIstored\fR strings. It should not be used for \fIquery\fR strings, increasing the chance that newly assigned characters work as expected. .Sp For further information on \fIstored\fR and \fIquery\fR strings, see RFC\ 3454, section\ 7. .RE .RS 4 .Sp The function returned can be called with a single parameter, the string to be prepared, and returns the prepared string. It will die if the input string cannot be successfully prepared because it would contain invalid output (so use \&\f(CW\*(C`eval\*(C'\fR if necessary). .Sp For performance reasons, it is strongly recommended to call the \&\f(CW\*(C`new\*(C'\fR function as few times as possible, i.\ e. exactly once per \&\fIstringprep\fR profile. It might also be better not to use this module directly but to use (or write) a module implementing a profile, such as Authen::SASL::SASLprep. .RE .SH "IMPLEMENTING PROFILES" .IX Header "IMPLEMENTING PROFILES" You can easily implement a \fIstringprep\fR profile without subclassing: .PP .Vb 1 \& package ACME::ExamplePrep; \& \& use Unicode::Stringprep; \& \& use Unicode::Stringprep::Mapping; \& use Unicode::Stringprep::Prohibited; \& \& *exampleprep = Unicode::Stringprep\->new( \& 3.2, \& [ \e@Unicode::Stringprep::Mapping::B1, ], \& \*(Aq\*(Aq, \& [ \e@Unicode::Stringprep::Prohibited::C12, \& \e@Unicode::Stringprep::Prohibited::C22, ], \& 1, \& ); .Ve .PP This binds \f(CW\*(C`ACME::ExamplePrep::exampleprep\*(C'\fR to the function created by \f(CW\*(C`Unicode::Stringprep\->new\*(C'\fR. .PP Usually, it is not necessary to subclass this module. Sublassing this module is not recommended. .SH "DATA TABLES" .IX Header "DATA TABLES" The following modules contain the data tables from RFC\ 3454. These modules are automatically loaded when loading \&\f(CW\*(C`Unicode::Stringprep\*(C'\fR. .IP \(bu 4 Unicode::Stringprep::Unassigned .Sp .Vb 1 \& @Unicode::Stringprep::Unassigned::A1 # Appendix A.1 .Ve .IP \(bu 4 Unicode::Stringprep::Mapping .Sp .Vb 3 \& @Unicode::Stringprep::Mapping::B1 # Appendix B.1 \& @Unicode::Stringprep::Mapping::B2 # Appendix B.2 \& @Unicode::Stringprep::Mapping::B2 # Appendix B.3 .Ve .IP \(bu 4 Unicode::Stringprep::Prohibited .Sp .Vb 11 \& @Unicode::Stringprep::Prohibited::C11 # Appendix C.1.1 \& @Unicode::Stringprep::Prohibited::C12 # Appendix C.1.2 \& @Unicode::Stringprep::Prohibited::C21 # Appendix C.2.1 \& @Unicode::Stringprep::Prohibited::C22 # Appendix C.2.2 \& @Unicode::Stringprep::Prohibited::C3 # Appendix C.3 \& @Unicode::Stringprep::Prohibited::C4 # Appendix C.4 \& @Unicode::Stringprep::Prohibited::C5 # Appendix C.5 \& @Unicode::Stringprep::Prohibited::C6 # Appendix C.6 \& @Unicode::Stringprep::Prohibited::C7 # Appendix C.7 \& @Unicode::Stringprep::Prohibited::C8 # Appendix C.8 \& @Unicode::Stringprep::Prohibited::C9 # Appendix C.9 .Ve .IP \(bu 4 Unicode::Stringprep::BiDi .Sp .Vb 2 \& @Unicode::Stringprep::BiDi::D1 # Appendix D.1 \& @Unicode::Stringprep::BiDi::D2 # Appendix D.2 .Ve .SH CAVEATS .IX Header "CAVEATS" In Unicode 3.2 to 4.0.1, the specification of UAX #15: Unicode Normalization Forms for forms NFC and NFKC is not logically self-consistent. This has been fixed in Corrigendum #5 (). .PP Unfortunately, this yields two ways to implement NFC and NFKC in Unicode 3.2, on which the Stringprep standard is based: one based on a literal interpretation of the original specification and one based on the corrected specification. The output of these implementations differs for a small class of strings, all of which can't appear in meaningful text. See UAX #15, section 19 for details. .PP This module will check for these strings and, if normalization is done, prohibit them in output as it is not possible to interoperate under these circumstandes. .PP Please note that due to this, the \fInormalization\fR step may cause the preparation to fail. That is, the preparation function may die even if there are no prohibited characters and no checks for bidi sequences and unassigned characters, which may be surprising. .SH AUTHOR .IX Header "AUTHOR" Claus Färber .SH LICENSE .IX Header "LICENSE" Copyright 2007\-2009 Claus Färber. .PP This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. .SH "SEE ALSO" .IX Header "SEE ALSO" Unicode::Normalize, RFC\ 3454 ()