About program launch scripts (.sh and .bat files) and portability 24 Jan 2018 GLF --------------------------------------------------------------------------------------------- The GLFPE (GLF Programming Environment) is intended to allow practically portable programming on multiple operating system environments. The way things work for a user of GLFPE is roughly similar on either Windows or Linux (specifically Debian Linux, chosen by the author). A variety of tools and techniques work similarly whether a command line shell or a GUI (graphical user interface) windowing environment is used to invoke those tools. TIME WAS... For years, in all versions of Windows and (almost) all versions of Linux desktop environments, the file manager (Explorer in Windows, often Nautilus or PCFileman or Caja in Linux) would allow execution of an executable file (.exe or .com in Windows) or command script (.sh in Linux or .bat in Windows) by clicking or double-clicking on it in the file manager window. That arrangement was very convenient for programmers testing a broad variety of programs, particularly windowed GUI programs. Look up the program in Explorer (or whatever) and click to execute, then get the program. Simple. TIMES HAVE CHANGED. Problem under Linux (including Raspberry Pi variant) ---------------------------------------------------- Partly because users (even programmers) often click the WRONG thing or things they SHOULD NOT CLICK from file manager windows, the wizards who design operating systems have begun to change things, mostly on Linux. In the current age of paranoia about malware, we must be protected from ourselves. The problem on Linux is a recent one. the author found out about this when converting from Debian 8 to Debian 9. Recent versions of the GCC compiler produce a different form of executable program than before, due to the standardization of a requirement that such programs compile to PIC (position independent code). This change was partly to address vulnerabilities in programs which compiled using absolute addressing where it was not necessary, and (almost always) strategically undesirable. The change was a GOOD THING. The problem is that file managers rely on certain lower-level OS facilities to identify the type of files when presenting them in their GUI windows, and the change to PIC had a side effect -- the current run of file managers do not see the compiled programs as "executable programs", they see them as "libraries', which they DO NOT LAUNCH. Furthermore, as told by the various discussion threads on various related OS forums, there is apparently no desire on the part of OS developers to change the current behavior back to the original. Though the author disagrees, the consensus found on the threads is that te file manager SHOULDN'T have that behavior anyway, EVEN THOUGH EVERY WINDOWED OS IN MODERN HISTORY FOR TWENTY FIVE YEARS has exhibited that behavior. Sigh. Before the newer scheme, which showed up in Debian 9, one could use the file manager, locate an executable program (usually having a distinctive icon in the GUI display), presumably just compiled by the user, and could bring it up for a test run by clicking it. NOW, that doesn't work. It would be REALLY INCONVENIENT to have to drop into a command shell, change directories, and invoke the program by name, especially if you do it a lot. Fortunately, the author came up with a workaround. The change in click-to-execute behavior has not (yet) extended to script files. You can still start any script file with executable flag set by clicking it in the file manager GUI. For each such program which won't launch itself, one can have a shell script (.sh file) which launches the actual program. This is actually better for testing, in that one can provide command line parameters which pass filenames and such to set up specific test data. For the GLFPE sample source code programs, the author has provided matching program launcher scripts to invoke the compiled programs from a GUI. By convention, for each program named for example "programename", two laucher scripts are written. One, "programname_launch.sh" is intended to bring up the program without extra command line options. A second script, "programname_test.sh", is similar to the first, but adds any useful sample test parameters to invoke a practical test run. Of course, any relevant test data referenced in the test script must also be provided. This scheme allows the user to work in a similar manner as before, except that instead of clicking on the executable program itself, the user clicks the launcher's icon. That works (for now). Problem under Windows --------------------- Actually, Windows behavior has not yet changed. The problem is self-imposed by the author's intent that the GLFPE for Windows is as portable as possible. The entire programming environment, source code, and toolset are assumed copied onto a USB stick or other external filesystem drive. It is NOT NECESSARY to install any of the GLFPE tools permanently to the base Windows computer environment (i.e. C: drive). This imposes certain restrictions on the way the tools and user-compiled programs may be run. Ordinarily, for installed programs under Windows, the requirements which individual programs may have to find their own 'parts", e.g. batch scripts , executable components, and DLLs, certain additions are made to the PATH variable, and perhaps other environment variables. That allows such installed programs to "just know" where to find their associated components, so when one double-clicks in Explorer to execute them, they find what they need from those environment variables and run properly. To make the GLFPE for Windows truly portable, installing to the host computer is undesirable, and should be minimized to ONLY those elements for which there is no alternative, such as hardware drivers, which must be always installed to the host. Ideally, all of the compilers, editors, and the compiled sample and user code should run without imposing other commitments upon the host OS. The main advantage of a portable external drive base for GLFPE is that a user may work on or demonstrate the same projects on multiple computers without forcing any of those computers to install anything (except maybe hardware drivers if the programs call for such). The author has provided launcher batch scripts to start up the key tools from an Explorer window looking at the pertinent USB flash (or other) drive. The user employs the interface provided by those tools (in some cases a command line) to navigate and find relevant files. Windows, and some valuable Windows programs, don't always make portability easy. The author (GLF) has come up with carefully structured batch scripts (.bat files) which can launch the various tools in the GLFPE for Windows toolset without (most of the time) making permanent changes to the host OS. The batch files make much use of RELATIVE pathnames (e.g. \glfpe\mgw481\bin instead of C:\glfpe\mgw481\bin) in the scripts to allow the programs to adapt to whatever drive letter on which they happen to find themselves (from C: to G:). The scripts each locally set the PATH variable to add the required paths to find executables and DLLs, and systematically change drives and directories internally so that the invoked program executes with all its required environment variables intact. The scope of the batch scripts is local, so none of the changes exist outside the running of whatever programs are invoked within it. Therefore launcher scripts have been provided to start the various tools, If the programs had been permanently installed into Windows, they might be available from a menu item in the Start menu, or by clicking a shortcut on the desktop. More importantly, file associations could be made so that clicking on a file with a particular extension (e.g..dev for Dev-C_++ projects) would bring up the program associated with that extension (e.g. Orwell Dev-C++). Because such associations tie to a specific application as installed IN A PARTICULAR LOCATION, that behavior is not truly portable. It is important to note that this restiction is self-imposed. Even though all the components of GLFPE for Windows CAN run portably from an external drive or USB stick, the user can copy the same files to the C: drive (or other internal drive) and run them from there. The provided launchers automatically search for the HIGHEST drive letter first, down to lower, to determine which drive letter holds the GLFPE files. If that drive is C:, the programs still work, just as portable implementations. However, the user is free to add permanent changes to PATH and other environment variables (manually), may create desktop shortcuts to launch the progras, and MAY USE FILE ASSOCIATIONS if the files will always be in the same place. NOTE: there are exceptions to full portability. Some GLFPE applications, notably the Arduino IDE and Orwell Dev-C++, cane have side effects due to internal hard-coding of expectations for where to store their configuration files. The standard location for installed Windows applications to store program-related information including internal configurations is to the Users folders (C:\Users\[username]\...). The Orwell program can be run either portably or assumed installed. Under some conditions, one might accidentally get nonportable behavior from Orwell and it will store and retrieve configurations from C: drive even if the program is NOT INSTALLED TO C:. The same is true of the Arduino IDE, except that it ALWAYS stores its conffigurations to C:, whether you like it or not. It will RUN portably from an arbitrary drive letter on USB stick, but it will lose track of its own preferences 9such as where to find the "sketches" or source code) between reboots or drive changes. It still works MOSTLY portably. The command-line tools (such as the GCC compiler) make no commitments to the host computer and run from any drive, but they do need to have their PATH and other variables preset before use, hence the launcher scripts.