.\" -*- mode: troff; coding: utf-8 -*-
.\" Automatically generated by Pod::Man 5.0102 (Pod::Simple 3.45)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>.
.ie n \{\
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "Mail::SpamAssassin::Plugin::ExtractText 3"
.TH Mail::SpamAssassin::Plugin::ExtractText 3 2024-09-01 "perl v5.40.0" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH NAME
ExtractText \- extracts text from documenmts.
.SH SYNOPSIS
.IX Header "SYNOPSIS"
loadplugin Mail::SpamAssassin::Plugin::ExtractText
.PP
ifplugin Mail::SpamAssassin::Plugin::ExtractText
.PP
.Vb 2
\&  extracttext_external  pdftotext  /usr/bin/pdftotext \-nopgbrk \-layout \-enc UTF\-8 {} \-
\&  extracttext_use       pdftotext  .pdf application/pdf
\&
\&  # http://docx2txt.sourceforge.net
\&  extracttext_external  docx2txt   /usr/bin/docx2txt {} \-
\&  extracttext_use       docx2txt   .docx application/docx
\&
\&  extracttext_external  antiword   /usr/bin/antiword \-t \-w 0 \-m UTF\-8.txt {}
\&  extracttext_use       antiword   .doc application/(?:vnd\e.?)?ms\-?word.*
\&
\&  extracttext_external  unrtf      /usr/bin/unrtf \-\-nopict {}
\&  extracttext_use       unrtf      .doc .rtf application/rtf text/rtf
\&
\&  extracttext_external  odt2txt    /usr/bin/odt2txt \-\-encoding=UTF\-8 {}
\&  extracttext_use       odt2txt    .odt .ott application/.*?opendocument.*text
\&  extracttext_use       odt2txt    .sdw .stw application/(?:x\-)?soffice application/(?:x\-)?starwriter
\&
\&  extracttext_external  tesseract  {OMP_THREAD_LIMIT=1} /usr/bin/tesseract \-c page_separator= {} \-
\&  extracttext_use       tesseract  .jpg .png .bmp .tif .tiff image/(?:jpeg|png|x\-ms\-bmp|tiff)
\&
\&  # QR\-code decoder
\&  extracttext_external  zbar       /usr/bin/zbarimg \-q \-D {}
\&  extracttext_use       zbar       .jpg .png .pdf image/(?:jpeg|png) application/pdf
\&
\&  add_header   all          ExtractText\-Flags _EXTRACTTEXTFLAGS_
\&  header       PDF_NO_TEXT  X\-ExtractText\-Flags =~ /\ebpdftotext_NoText\eb/
\&  describe     PDF_NO_TEXT  PDF without text
\&  score        PDF_NO_TEXT  0.001
\&
\&  header       DOC_NO_TEXT  X\-ExtractText\-Flags =~ /\eb(?:antiword|openxml|unrtf|odt2txt)_NoText\eb/
\&  describe     DOC_NO_TEXT  Document without text
\&  score        DOC_NO_TEXT  0.001
\&
\&  header       EXTRACTTEXT  exists:X\-ExtractText\-Flags
\&  describe     EXTRACTTEXT  Email processed by extracttext plugin
\&  score        EXTRACTTEXT  0.001
.Ve
.PP
endif
.SH DESCRIPTION
.IX Header "DESCRIPTION"
This module uses external tools to extract text from message parts,
and then sets the text as the rendered part. External tool must output
plain text, not HTML or other non-textual result.
.PP
How to extract text is completely configurable, and based on
MIME part type and file name.
.SH CONFIGURATION
.IX Header "CONFIGURATION"
All configuration lines in user_prefs files will be ignored.
.IP "extracttext_maxparts (default: 10)" 4
.IX Item "extracttext_maxparts (default: 10)"
Configure the maximum mime parts number to analyze, a value of 0 means all mime parts
will be analyzed
.IP "extracttext_timeout (default: 5 10)" 4
.IX Item "extracttext_timeout (default: 5 10)"
Configure the timeout in seconds of external tool checks, per attachment.
.Sp
Second argument speficies maximum total time for all checks.
.SS Tools
.IX Subsection "Tools"
.IP extracttext_use 4
.IX Item "extracttext_use"
Specifies what tool to use for what message parts.
.Sp
The general syntax is
.Sp
extracttext_use  \f(CW\*(C`name\*(C'\fR  \f(CW\*(C`specifiers\*(C'\fR
.IP name 4
.IX Item "name"
the internal name of a tool.
.IP specifiers 4
.IX Item "specifiers"
File extension and regular expressions for file names and MIME
types. The regular expressions are anchored to beginning and end.
.PP
\fIExamples\fR
.IX Subsection "Examples"
.PP
.Vb 4
\&        extracttext_use  antiword  .doc application/(?:vnd\e.?)?ms\-?word.*
\&        extracttext_use  openxml   .docx .dotx .dotm application/(?:vnd\e.?)openxml.*?word.*
\&        extracttext_use  openxml   .doc .dot application/(?:vnd\e.?)?ms\-?word.*
\&        extracttext_use  unrtf     .doc .rtf application/rtf text/rtf
.Ve
.IP extracttext_external 4
.IX Item "extracttext_external"
Defines an external tool.  The tool must read a document on standard input
or from a file and write text to standard output.
.Sp
The special keyword "{}" will be substituted at runtime with the temporary
filename to be scanned by the external tool.
.Sp
Environment variables can be defined with "{KEY=VALUE}", these strings will
be removed from commandline.
.Sp
It is required that commandline used outputs result directly to STDOUT.
.Sp
The general syntax is
.Sp
extracttext_external \f(CW\*(C`name\*(C'\fR \f(CW\*(C`command\*(C'\fR \f(CW\*(C`parameters\*(C'\fR
.IP name 4
.IX Item "name"
The internal name of this tool.
.IP command 4
.IX Item "command"
The full path to the external command to run.
.IP parameters 4
.IX Item "parameters"
Parameters for the external command. The temporary file name containing
the document will be automatically added as last parameter.
.PP
\fIExamples\fR
.IX Subsection "Examples"
.PP
.Vb 3
\&        extracttext_external  antiword  /usr/bin/antiword \-t \-w 0 \-m UTF\-8.txt {} \-
\&        extracttext_external  unrtf     /usr/bin/unrtf \-\-nopict {}
\&        extracttext_external  odt2txt   /usr/bin/odt2txt \-\-encoding=UTF\-8 {}
.Ve
.SS Metadata
.IX Subsection "Metadata"
The plugin adds some pseudo headers to the message. These headers are seen by
the bayes system, and can be used in normal SpamAssassin rules.
.PP
The headers are also available as template tags as noted below.
.PP
\fIExample\fR
.IX Subsection "Example"
.PP
The fictional example headers below are based on a message containing this:
.IP "1 A perfectly normal PDF." 4
.IX Item "1 A perfectly normal PDF."
.PD 0
.IP "2 An OpenXML document with a word document inside. Neither Office document contains text." 4
.IX Item "2 An OpenXML document with a word document inside. Neither Office document contains text."
.PD
.PP
\fIHeaders\fR
.IX Subsection "Headers"
.IP X\-ExtractText-Chars 4
.IX Item "X-ExtractText-Chars"
Tag: _EXTRACTTEXTCHARS_
.Sp
Contains a count of characters that were extracted.
.Sp
X\-ExtractText-Chars: 10970
.IP X\-ExtractText-Words 4
.IX Item "X-ExtractText-Words"
Tag: _EXTRACTTEXTWORDS_
.Sp
Contains a count of "words" that were extracted.
.Sp
X\-ExtractText-Chars: 1599
.IP X\-ExtractText-Tools 4
.IX Item "X-ExtractText-Tools"
Tag: _EXTRACTTEXTTOOLS_
.Sp
Contains chains of tools used for extraction.
.Sp
X\-ExtractText-Tools: pdftotext openxml_antiword
.IP X\-ExtractText-Types 4
.IX Item "X-ExtractText-Types"
Tag: _EXTRACTTEXTTYPES_
.Sp
Contains chains of MIME types for parts found during extraction.
.Sp
X\-ExtractText-Types: application/pdf; application/vnd.openxmlformats\-officedocument.wordprocessingml.document, application/ms\-word
.IP X\-ExtractText-Extensions 4
.IX Item "X-ExtractText-Extensions"
Tag: _EXTRACTTEXTEXTENSIONS_
.Sp
Contains chains of canonicalized file extensions for parts
found during extraction.
.Sp
X\-ExtractText-Extensions: pdf docx
.IP X\-ExtractText-Flags 4
.IX Item "X-ExtractText-Flags"
Tag: _EXTRACTTEXTFLAGS_
.Sp
Contains notes from the plugin.
.Sp
X\-ExtractText-Flags: openxml_NoText
.IP X\-ExtractText-Uris 4
.IX Item "X-ExtractText-Uris"
Tag: _EXTRACTTEXTURIS_
.Sp
Contains uris extracted from the plugin.
.Sp
X\-ExtractText-Uris: https://spamassassin.apache.org
.PP
\fIRules\fR
.IX Subsection "Rules"
.PP
Example:
.PP
.Vb 2
\&        header    PDF_NO_TEXT  X\-ExtractText\-Flags =~ /\ebpdftotext_Notext\eb/
\&        describe  PDF_NO_TEXT  PDF without text
.Ve