; The flavor perl__.exe is required if you want to start your program without a VIO window present, but not "detach"ed (run "help detach" for more info). Very useful for extensions which use PM, like "Perl/Tk" or "OpenGL". Note also that the differences between PM and VIO executables are only in the default behaviour. One can start any executable in any kind of session by using the arguments "/fs", "/pm" or "/win" switches of the command "start" (of CMD.EXE or a similar shell). Alternatively, one can use the numeric first argument of the "system" Perl function (see OS2::Process). perl___.exe This is an "omf"-style executable which is dynamically linked to perl.dll and CRT DLL. I know no advantages of this executable over "perl.exe", but it cannot fork() at all. Well, one advantage is that the build process is not so convoluted as with "perl.exe". It is a VIO application. Why strange names? Since Perl processes the "#!"-line (cf. "DESCRIPTION" in perlrun, "Command Switches" in perlrun, "No Perl script found in input" in perldiag), it should know when a program is a Perl. There is some naming convention which allows Perl to distinguish correct lines from wrong ones. The above names are almost the only names allowed by this convention which do not contain digits (which have absolutely different semantics). Why dynamic linking? Well, having several executables dynamically linked to the same huge library has its advantages, but this would not substantiate the additional work to make it compile. The reason is the complicated-to- developers but very quick and convenient-to-users "hard" dynamic linking used by OS/2. There are two distinctive features of the dyna-linking model of OS/2: first, all the references to external functions are resolved at the compile time; second, there is no runtime fixup of the DLLs after they are loaded into memory. The first feature is an enormous advantage over other models: it avoids conflicts when several DLLs used by an application export entries with the same name. In such cases "other" models of dyna-linking just choose between these two entry points using some random criterion - with predictable disasters as results. But it is the second feature which requires the build of perl.dll. The address tables of DLLs are patched only once, when they are loaded. The addresses of the entry points into DLLs are guaranteed to be the same for all the programs which use the same DLL. This removes the runtime fixup - once DLL is loaded, its code is read-only. While this allows some (significant?) performance advantages, this makes life much harder for developers, since the above scheme makes it impossible for a DLL to be "linked" to a symbol in the .EXE file. Indeed, this would need a DLL to have different relocations tables for the (different) executables which use this DLL. However, a dynamically loaded Perl extension is forced to use some symbols from the perl executable, e.g., to know how to find the arguments to the functions: the arguments live on the perl internal evaluation stack. The solution is to put the main code of the interpreter into a DLL, and make the .EXE file which just loads this DLL into memory and supplies command-arguments. The extension DLL cannot link to symbols in .EXE, but it has no problem linking to symbols in the .DLL. This greatly increases the load time for the application (as well as complexity of the compilation). Since interpreter is in a DLL, the C RTL is basically forced to reside in a DLL as well (otherwise extensions would not be able to use CRT). There are some advantages if you use different flavors of perl, such as running perl.exe and perl__.exe simultaneously: they share the memory of perl.dll. NOTE. There is one additional effect which makes DLLs more wasteful: DLLs are loaded in the shared memory region, which is a scarse resource given the 512M barrier of the "standard" OS/2 virtual memory. The code of .EXE files is also shared by all the processes which use the particular .EXE, but they are "shared in the private address space of the process"; this is possible because the address at which different sections of the .EXE file are loaded is decided at compile-time, thus all the processes have these sections loaded at same addresses, and no fixup of internal links inside the .EXE is needed. Since DLLs may be loaded at run time, to have the same mechanism for DLLs one needs to have the address range of any of the loaded DLLs in the system to be available in all the processes which did not load a particular DLL yet. This is why the DLLs are mapped to the shared memory region. Why chimera build? Current EMX environment does not allow DLLs compiled using Unixish "a.out" format to export symbols for data (or at least some types of data). This forces "omf"-style compile of perl.dll. Current EMX environment does not allow .EXE files compiled in "omf" format to fork(). fork() is needed for exactly three Perl operations: o explicit fork() in the script, o "open FH, "|-"" o "open FH, "-|"", in other words, opening pipes to itself. While these operations are not questions of life and death, they are needed for a lot of useful scripts. This forces "a.out"-style compile of perl.exe. ENVIRONMENT Here we list environment variables with are either OS/2- and DOS- and Win*-specific, or are more important under OS/2 than under other OSes. "PERLLIB_PREFIX" Specific for EMX port. Should have the form path1;path2 or path1 path2 If the beginning of some prebuilt path matches path1, it is substituted with path2. Should be used if the perl library is moved from the default location in preference to "PERL(5)LIB", since this would not leave wrong entries in @INC. For example, if the compiled version of perl looks for @INC in f:/perllib/lib, and you want to install the library in h:/opt/gnu, do set PERLLIB_PREFIX=f:/perllib/lib;h:/opt/gnu This will cause Perl with the prebuilt @INC of f:/perllib/lib/5.00553/os2 f:/perllib/lib/5.00553 f:/perllib/lib/site_perl/5.00553/os2 f:/perllib/lib/site_perl/5.00553 . to use the following @INC: h:/opt/gnu/5.00553/os2 h:/opt/gnu/5.00553 h:/opt/gnu/site_perl/5.00553/os2 h:/opt/gnu/site_perl/5.00553 . "PERL_BADLANG" If 0, perl ignores setlocale() failing. May be useful with some strange locales. "PERL_BADFREE" If 0, perl would not warn of in case of unwarranted free(). With older perls this might be useful in conjunction with the module DB_File, which was buggy when dynamically linked and OMF-built. Should not be set with newer Perls, since this may hide some real problems. "PERL_SH_DIR" Specific for EMX port. Gives the directory part of the location for sh.exe. "USE_PERL_FLOCK" Specific for EMX port. Since flock(3) is present in EMX, but is not functional, it is emulated by perl. To disable the emulations, set environment variable "USE_PERL_FLOCK=0". "TMP" or "TEMP" Specific for EMX port. Used as storage place for temporary files. Evolution Here we list major changes which could make you by surprise. Text-mode filehandles Starting from version 5.8, Perl uses a builtin translation layer for text-mode files. This replaces the efficient well-tested EMX layer by some code which should be best characterized as a "quick hack". In addition to possible bugs and an inability to follow changes to the translation policy with off/on switches of TERMIO translation, this introduces a serious incompatible change: before sysread() on text-mode filehandles would go through the translation layer, now it would not. Priorities "setpriority" and "getpriority" are not compatible with earlier ports by Andreas Kaiser. See "setpriority, getpriority". DLL name mangling: pre 5.6.2 With the release 5.003_01 the dynamically loadable libraries should be rebuilt when a different version of Perl is compiled. In particular, DLLs (including perl.dll) are now created with the names which contain a checksum, thus allowing workaround for OS/2 scheme of caching DLLs. It may be possible to code a simple workaround which would o find the old DLLs looking through the old @INC; o mangle the names according to the scheme of new perl and copy the DLLs to these names; o edit the internal "LX" tables of DLL to reflect the change of the name (probably not needed for Perl extension DLLs, since the internally coded names are not used for "specific" DLLs, they used only for "global" DLLs). o edit the internal "IMPORT" tables and change the name of the "old" perl????.dll to the "new" perl????.dll. DLL name mangling: 5.6.2 and beyond In fact mangling of extension DLLs was done due to misunderstanding of the OS/2 dynaloading model. OS/2 (effectively) maintains two different tables of loaded DLL: Global DLLs those loaded by the base name from "LIBPATH"; including those associated at link time; specific DLLs loaded by the full name. When resolving a request for a global DLL, the table of already-loaded specific DLLs is (effectively) ignored; moreover, specific DLLs are always loaded from the prescribed path. There is/was a minor twist which makes this scheme fragile: what to do with DLLs loaded from "BEGINLIBPATH" and "ENDLIBPATH" (which depend on the process) . from "LIBPATH" which effectively depends on the process (although "LIBPATH" is the same for all the processes). Unless "LIBPATHSTRICT" is set to "T" (and the kernel is after 2000/09/01), such DLLs are considered to be global. When loading a global DLL it is first looked in the table of already-loaded global DLLs. Because of this the fact that one executable loaded a DLL from "BEGINLIBPATH" and "ENDLIBPATH", or . from "LIBPATH" may affect which DLL is loaded when another executable requests a DLL with the same name. This is the reason for version-specific mangling of the DLL name for perl DLL. Since the Perl extension DLLs are always loaded with the full path, there is no need to mangle their names in a version-specific ways: their directory already reflects the corresponding version of perl, and @INC takes into account binary compatibility with older version. Starting from 5.6.2 the name mangling scheme is fixed to be the same as for Perl 5.005_53 (same as in a popular binary release). Thus new Perls will be able to resolve the names of old extension DLLs if @INC allows finding their directories. However, this still does not guarantee that these DLL may be loaded. The reason is the mangling of the name of the Perl DLL. And since the extension DLLs link with the Perl DLL, extension DLLs for older versions would load an older Perl DLL, and would most probably segfault (since the data in this DLL is not properly initialized). There is a partial workaround (which can be made complete with newer OS/2 kernels): create a forwarder DLL with the same name as the DLL of the older version of Perl, which forwards the entry points to the newer Perl's DLL. Make this DLL accessible on (say) the "BEGINLIBPATH" of the new Perl executable. When the new executable accesses old Perl's extension DLLs, they would request the old Perl's DLL by name, get the forwarder instead, so effectively will link with the currently running (new) Perl DLL. This may break in two ways: o Old perl executable is started when a new executable is running has loaded an extension compiled for the old executable (ouph!). In this case the old executable will get a forwarder DLL instead of the old perl DLL, so would link with the new perl DLL. While not directly fatal, it will behave the same as new executable. This beats the whole purpose of explicitly starting an old executable. o A new executable loads an extension compiled for the old executable when an old perl executable is running. In this case the extension will not pick up the forwarder - with fatal results. With support for "LIBPATHSTRICT" this may be circumvented - unless one of DLLs is started from . from "LIBPATH" (I do not know whether "LIBPATHSTRICT" affects this case). REMARK. Unless newer kernels allow . in "BEGINLIBPATH" (older do not), this mess cannot be completely cleaned. (It turns out that as of the beginning of 2002, . is not allowed, but .\. is - and it has the same effect.) REMARK. "LIBPATHSTRICT", "BEGINLIBPATH" and "ENDLIBPATH" are not environment variables, although cmd.exe emulates them on "SET ..." lines. From Perl they may be accessed by Cwd::extLibpath and Cwd::extLibpath_set. DLL forwarder generation Assume that the old DLL is named perlE0AC.dll (as is one for 5.005_53), and the new version is 5.6.1. Create a file perl5shim.def-leader with LIBRARY 'perlE0AC' INITINSTANCE TERMINSTANCE DESCRIPTION '@#perl5-porters@perl.org:5.006001#@ Perl module for 5.00553 -> Perl 5.6.1 forwarder' CODE LOADONCALL DATA LOADONCALL NONSHARED MULTIPLE EXPORTS modifying the versions/names as needed. Run perl -wnle "next if 0../EXPORTS/; print qq( \"$1\") if /\"(\w+)\"/" perl5.def >lst in the Perl build directory (to make the DLL smaller replace perl5.def with the definition file for the older version of Perl if present). cat perl5shim.def-leader lst >perl5shim.def gcc -Zomf -Zdll -o perlE0AC.dll perl5shim.def -s -llibperl (ignore multiple "warning L4085"). Threading As of release 5.003_01 perl is linked to multithreaded C RTL DLL. If perl itself is not compiled multithread-enabled, so will not be perl's malloc(). However, extensions may use multiple thread on their own risk. This was needed to compile "Perl/Tk" for XFree86-OS/2 out-of-the-box, and link with DLLs for other useful libraries, which typically are compiled with "-Zmt -Zcrtdll". Calls to external programs Due to a popular demand the perl external program calling has been changed wrt Andreas Kaiser's port. If perl needs to call an external program via shell, the f:/bin/sh.exe will be called, or whatever is the override, see ""PERL_SH_DIR"". Thus means that you need to get some copy of a sh.exe as well (I use one from pdksh). The path F:/bin above is set up automatically during the build to a correct value on the builder machine, but is overridable at runtime, Reasons: a consensus on "perl5-porters" was that perl should use one non-overridable shell per platform. The obvious choices for OS/2 are cmd.exe and sh.exe. Having perl build itself would be impossible with cmd.exe as a shell, thus I picked up "sh.exe". This assures almost 100% compatibility with the scripts coming from *nix. As an added benefit this works as well under DOS if you use DOS-enabled port of pdksh (see "Prerequisites"). Disadvantages: currently sh.exe of pdksh calls external programs via fork()/exec(), and there is no functioning exec() on OS/2. exec() is emulated by EMX by an asynchronous call while the caller waits for child completion (to pretend that the "pid" did not change). This means that 1 extra copy of sh.exe is made active via fork()/exec(), which may lead to some resources taken from the system (even if we do not count extra work needed for fork()ing). Note that this a lesser issue now when we do not spawn sh.exe unless needed (metachars found). One can always start cmd.exe explicitly via system 'cmd', '/c', 'mycmd', 'arg1', 'arg2', ... If you need to use cmd.exe, and do not want to hand-edit thousands of your scripts, the long-term solution proposed on p5-p is to have a directive use OS2::Cmd; which will override system(), exec(), ``, and "open(,'...|')". With current perl you may override only system(), readpipe() - the explicit version of ``, and maybe exec(). The code will substitute the one- argument call to system() by "CORE::system('cmd.exe', '/c', shift)". If you have some working code for "OS2::Cmd", please send it to me, I will include it into distribution. I have no need for such a module, so cannot test it. For the details of the current situation with calling external programs, see "Starting OS/2 (and DOS) programs under Perl". Set us mention a couple of features: o External scripts may be called by their basename. Perl will try the same extensions as when processing -S command-line switch. o External scripts starting with "#!" or "extproc " will be executed directly, without calling the shell, by calling the program specified on the rest of the first line. Memory allocation Perl uses its own malloc() under OS/2 - interpreters are usually malloc-bound for speed, but perl is not, since its malloc is lightning- fast. Perl-memory-usage-tuned benchmarks show that Perl's malloc is 5 times quicker than EMX one. I do not have convincing data about memory footprint, but a (pretty random) benchmark showed that Perl's one is 5% better. Combination of perl's malloc() and rigid DLL name resolution creates a special problem with library functions which expect their return value to be free()d by system's free(). To facilitate extensions which need to call such functions, system memory-allocation functions are still available with the prefix "emx_" added. (Currently only DLL perl has this, it should propagate to perl_.exe shortly.) Threads One can build perl with thread support enabled by providing "-D usethreads" option to Configure. Currently OS/2 support of threads is very preliminary. Most notable problems: "COND_WAIT" may have a race condition (but probably does not due to edge- triggered nature of OS/2 Event semaphores). (Needs a reimplementation (in terms of chaining waiting threads, with the linked list stored in per-thread structure?)?) os2.c has a couple of static variables used in OS/2-specific functions. (Need to be moved to per-thread structure, or serialized?) Note that these problems should not discourage experimenting, since they have a low probability of affecting small programs. BUGS This description is not updated often (since 5.6.1?), see ./os2/Changes for more info. AUTHOR Ilya Zakharevich, cpan@ilyaz.org SEE ALSO perl(1). perl v5.38.2 2024-02-11 PERLOS2(1perl)