ISSUE Live Boot and persistence 14 March 2022 GLF -------------------------------------------------------------------------------- "Vegas Rules" and security ---------------------------- GLF has been producing Live Boot custom remix Linux Operating Systems for over 10 years. GLF has developed a particular approach to Live Boot OS -- Vegas Rules: "Whatever happens in Vegas, stays in Vegas". That is, if you boot an OS from a Live Boot USB stick or DVD, the entire session is temporary. Since everything that happens during a session happens in RAM or the pseudo-hard-drive provided by RAMdisk, when the system is shut down, no unintended trace remains. The next time the same Live Boot medium is used, the experience is the same -- nothing done before affects the boot, the installed apps, the configuration -- it's all fresh from scratch every time. GLF has seen that as a security advantage for the most part. One cannot (without some effort) accidentally "mess up" the operating system by a misconfiguration, a bad installation of software, accidentally deleting Linux OS files, etc. Just reboot and all is forgiven. That is an advantage when testing new (to you) software, or test-installing one of the thousands of available apps in the Debian repository. GLF-generated live boot ISO images have an added security feature. By default all Debian live-build generated images have the same user name and password, known worldwide (User: user, PW: live). GLF has added an autostart script to each image which sets up a one-time dialog encouraging changing the password. It only takes a few seconds to change the password for each live boot session, logout and log back in. Because of "Vegas Rules", the new password is only valid for that live session, and can be arbitrarily changed from session to session. It is ordinarily only needed if the user logs out or when running some "superuser" (root-owned) operations. There are downsides to this amnesiac approach. If you try installing some software and you like it, every time you want it you need to reinstall it, at least until the next custom rebuild cycle, when you have an opportunity to alter the build specifications and add the new package to the list. Also, any configuration changes you WANT to keep across boots need to be saved in non-obvious and technically challenging ways, again to be included into the next updated live-build configuration. Of course, if you DON'T like the software, or if it causes some other problem, "Vegas Rules" mean the new software is gone after the next reboot -- problem solved. GLF has adapted within these constraints over the years, stubbornly pushing to keep the remix OSes he produced to remain noncommittal and fresh-start every time, while including more and more features into the base build images. He has built in successful workarounds for some of the bigger problems with live boot: where to put data so that it doesn't rob you of RAM and RAMdisk, how to provide for nondefault passwords for online security, etc. However, it is incumbent upon the user to manually ensure any necessary copies and backups of important work in progress or it gets lost with the reboot. GLF has addressed this personally by making liberal use of external drive storage. For many everyday tasks this is sufficient. There is another way -- persistence ------------------------------------- Users who are familiar with other live boot remix distributions are most likely aware that it is possible to add "persistence" to the live boot session with most Linux live OSes. It is possible to set up the live boot medium such that though the boot starts from a read-only medium, it ACTS as if it is changeable and PERSISTENT, that is, the data from one session is retained to the next. Edited documents in the home folder, configuration changes, installations of (a few) apps, all are kept and updated with each use. This is limited in that some kinds of updates, such as core OS images, cannot be upgraded this way. However, the ability to save a persistent user password, game save states, and working drafts of documents from session to session is an important feature for many users. Persistent storage with live boot helps take the backup burden off the user, since for the most part saving the files to home folder is sufficient to retain it for next time. GLF has been experimenting of late with the various technical ramifications of supporting live boot persistence in the least cumbersome, yet most secure way. If persistence is NOT wanted, it doesn't happen, but if it IS wanted, it's automatic. The Debian support for persistence is not difficult, but several steps must be taken for it to happen. There MUST be a properly identified drive partition SOMEWHERE IN THE SYSTEM for persistent data to be stored. This is not unlike the non-persistent methodology used by GLF to store important work on external USB flash or protable hard drives. However, the "persistence" scheme is automatic for live boot IF SPECIFIED PROPERLY. The nature of the live boot RAMdisk system is important here. The base filesystem for the live boot session comes off read-only medium (it could be a DVD). The session provides the APPEARANCE of a read-write filesystem to the user by providing an OVERLAY to the read-only file data using th RAM and a special overlay filesystem which retains only CHANGES in an invisible way. The user sees data being changed and saved as if it were only in one place, but in reality it could be in several chains of data on top of one another. That only works if the data doesn't accumulate too many small changes, such as filling up rapidly changing cache files. Most live boot sessions do not involve installation of new software, and most of the time, even without persistence enabled overlays work well for hours at a time. The persistence scheme adds another overlay layer in coordinated files on the identified persistence partition. These files are generally not useful by themselves, but are used in conjunction with the underlying overlays in the live boot filesystem. While booted, any persistent data visible in a session may be copied to another medium as normal. However, the data on a persistence partition is not usually meaningful or handled directly by the user of the associated live boot image. NOTE THAT a given persistence partition is tied to a particular live boot ISO image. The way the scheme works, it is possible for the persistence partition to be on a separate writeable medium, such as a second USB stick separate from the boot medium. If a given persistence partition is overlaid on top of the WRONG ISO image, data and/or configurations could be corrupted, rendering the "persistence" feature void and possibly losing data for the intended user. It is important that a persistence partition be coordinated with the underlying live boot image to prevent this corruption. Ideally, the persistence partition for a given live boot image will be an extra partition on the same boot medium. This is easily possible for USB flash drive boot media, but not possible for read-only DVD boot media, in which case a separate persistence medium can make even a DVD live session "writeable". Details -------- The persistence partition may be a FAT32 partition, or it may be an EXT4 (Linux) partition. Some programs which copy live boot images to USB use FAT32 for persistence (such as Universal USB Installer or UUI). The assumption by UUI and similar programs is that the persistence partition is another partition on the same USB drive used for the live boot image. Some older computers did not recognize USB boot media unless the boot partion was on FAT32. However, the modern Hybrid ISO images produced for Debian live boot are effectively CDROM/DVD images with an extra UEFI partition to allow booting either from Legacy BIOS (16-bit) systems or newer UEFI (32-bit) systems. Copying the ISO image directly to USB results in a read-only medium WITHOUT a partition table, and is not capable of accepting an additional 'persistence" partition. What to do... GLF has experimentally confirmed that the persistence partition (tested with an EXT4 partition) does NOT need to be on the same physical drive as the boot medium. It just needs to follow the other rules. A second USB stick may be formatted as EXT4, labeled "persistence" as required, and holding in the root folder a file "persistence.conf" with the single line of text data, "/ union". Assuming the bott is specified correctly, the partition will be automatically detected on whichever drive it exists, and will be overlaid with the live boot filesystem. The key detail to get persistence from live boot is the boot parameter "persistence" must be specified at boot, either frm the ISOLINUX DVD-style boot, or from the EFI boot started by GRUB on a UEFI boot system. The user MIGHT need to manually specify this extra parameter. On an ISOLINUX style boot, TAB will open the boot parameter text for editing. On EFI, entering 'e' before boot will similarly allow editing the parameter list. Generally, add the word "persistence" after (or in place of) the word "splash". If you then boot with that "persistence" parameter specified AND there is a drive SOMEWHERE in the drive chain (a USB slot?) which meets the specification (EXT4, labeled "persistence", contains "persistence.conf" with "/ union" in first line), then when the boot finishes, the persistence data will automatically be overlaid. If one has, say booted before with persistence and changed the password, that changed password should still be in effect. If there was a word processing document edited into the home folder, it will show up now with changes intact. One important consideration which has been verified by GLF -- if the boot parameter list includes "persistence" but there IS NOT a qualified persistence drive present, the boot will proceed normally without incident, except that it will be a fresh live boot as if no persistence were ever intended. GLF plans to use that feature for future GLF live boot configurations. The "persistence" boot parameter will be included BY DEFAULT, making the determining factor whether or not the persistence drive is present (the boot parameter specs are in the "build" live-build config file). That will give the user control over the mode of boot session. If there is no persistence drive or partition, there is no persistence, and "Vegas Rules" still apply. The user may retain a persistence USB stick inserted along with the boot medium at boot time, and the session will be persistent. Alternatively, omit the persistence stick, and the session will be volatile ("Vegas Rules"). How to make persistence automatic on a USB boot medium -------------------------------------------------------- In order to provide persistence support on a single USB stick, it is trickier, but GLF has proven a possible method. There are several methods available for creation of USB bootable media. The author suggests, and tends to use, the gnome-disk-utility package ("Disks" on the menu) as a practical way to copy ISO images to USB. That method does not by itself pprovide persistence, but does produce reliable boot media which also work well for running permant installations. Another method used by the author in the past is the Universal USB Installer (UUI), a Windows-only orgram available from www.pendrivelinux.com. If the UUI program is used, a menu selection MAYBE allows creation of a single USB stick capable of booting with persistence. UUI expects a FAT32 formatted drive as boot medium. The restrictions which apply to FAT32 get in the way -- no file may be larger than 4 Gb, and on many (especially GLF) ISO images, the data is larger than that. It also could prevent large files from working properly in the overlays used for persistence, which could break the storage of data. There are other alternatives for producing bootable USB sticks which allow user selection from a menu of bootable live images, which is convenient for trying out alternative OSes, or occasional use of specialty OSes such as Clonezilla for backing uo hard drives. GLF provides support for two of these in all GLF-generated ISO images: "multibootusb" and "ventoy" (there are other products or methods, but they tend to be Windows programs). The multiboot methods can be "bent" to cover persistence by simply limiting the choice to a single boot image, and providing an additional persistence partition. One problem with using multiboot to support persistence is that the persistence partition is meaningful only in conjunction with one ISO image. If one were to use a multiboot scheme to boot the WRONG ISO, it would probably corrupt the data on the persistence partition, making it useless for its original purpose. Therefore for a multiboot scheme to support persistence, it effectively forces use with a single ISO, not multiboot. Suggested methods to create boot disks ---------------------------------------- The preferred Linux-based methods used by GLF to create USB boot media so far have been: 1) Use the "Brasero" utility to burn te ISO image to DVD+R medium. This only works if the ISO file is smaller than 4.7Gb. It is possible to get persistence using the dual-disk method described below. 2) Use gnome-disk-utility to image the ISO directly to USB This is fast and works best under normal circumstances for single-boot media, but the drive becomes read-only and cannot hold a persistence partition. It is important to use this format if the boot medium will be used to install the OS permanently to a computer's hard drive. It is possible to get persistence using the dual-disk method described below. 3) In the past GLF created multiboot USB sticks using "multibootusb", a package which copies the boot data to a bootable FAT32 drive, expanding the ISO contents into readable files. That scheme could accommodate a second partition (EXT4 or FAT32) for persistence. However, the primary problem with multibootusb is that it copies to FAT32, and the LARGEST file within the live boot image (/live/filesystem.squashfs) is now often larger than the 4Gb FAT32 file size limit. Also, attempted permanent installations fail when booted this way -- the installer cannot "find" the boot medium "DVD" (hence the need for media created under (1) above). It could still work for persistence with small ISO images if a second EXT4 partition can be created alongside the bootable FAT32 partition. 4) The current GLF recommended method for multiboot media relies on "Ventoy", a utility which formats a USB stick with two partitions, a large exFAT partition and a small bootable EXT4 partition which holds a GRUB boot which lists all the ISO files present on the exFAT partition, and allows a user to select one, then boots from it. The exFAT format does not limit file sizes to 4Gb like FAT32, so the scheme accommodates the current large ISO images produced by GLF. GLF has experimentally proven that by shrinking the exFAT parttion to leave some free space, formatting that space to EXT4, labelling it "persistence", and copying/editing a "persistence.conf" file to it as specified, the Ventoy multiboot will support persistent live boot from a single USB stick. Creating a single-disk bootable ISO image with persistence ------------------------------------------------------------ The exFAT support in Linux is currently in transition from the older "exfat-fuse" and "exfat-utils" packages to the "exfatprogs" package which now applies for newer Linux kernels, since exFAT support was recently built into the kernel. Usually the most important utility in Linux for formatting and partitioning drives is "Gparted", but it has yet to catch up. Gparted/parted in the current Debian 11 "bullseye" does NOT WORK with exFAT partitions (Gparted 1.2.0-1). The unstable "sid" branch has an update which does work with exFAT, but as stated, "unstable". The gnome-disk-utility does not yet allow formatting to exFAT directly. However, GLF has found the gnome-disk-utility ("Disks") to be a currently valid, practical way to create such persistence support. In order to add persistence to a USB boot medium, GLF has adapted "Ventoy": 1) Create a "Ventoy" USB stick using the included menu item (Applications, System, Ventoy). GLF provides a front-end launcher which allows the user to see all USB media and select a drive for writing the Ventoy system (formatting the drive and making it bootable). The drive must be present but unmounted. BE CAREFUL not to select the wrong drive -- don't format your hard drive! 2) After successfully producing a Ventoy disk, the the specified drive will contain two partitions -- an exFAT partition labeled "ventoy" and a bootable FAT32 partition containing the EFI boot system and GRUB. The exFAT partition starts out empty of files. The user is expected to copy any desired ISO files to that partition, and they will show up in a boot menu when the USB medium is later used to boot. 3) To make a persistent storage medium, one must shrink the exFAT partition to accommodate extra space, and format the extra space as a valid EXT4 persistence partition. Use "Disks" to select the Ventoy USB medium. You should see the two partitions taking up all the disk space. Ensure all the partit ions are unmounted (turn "off" each one with the square button). Select the exFAT partition and DELETE IT (the "-" button in the menu). Then reselt the same space, and add a NEW PARTITION (the "+" botton). This time, slide the slider control left to limit the space to something a little larger than the intended ISO file to which persistence will apply. That will help prevent accidentally mismatching the poersistence overlay with the wrong ISO. Label the drive "ventoy". Format the drive, selecting "other", then "exFAT". When that is done, select the remaining empty space, create another new partition, label it "persistence", and format it EXT4. 4) Once the above is complete, the drive can be remounted, and both the "ventoy" and "persistence" partitions should show up in the file manager, and both should be empty. Copy the ISO file of interest to the "ventoy" partition, and create a new document into the "persistence" partition named "persistence.conf". Edit that document, including "/ union" as the first line of text (7 characters), and save it. The above multiboot stick will only have one ISO image to load, and it will now boot to assume persistence. The user can disable persistence temporarily by any of these methods, each of which breaks the chain of requirements for persistence: Edit "persistence" OUT of the boot parameter list at boot time (TAB or 'e'). The boot will mount the persistence partition as a normal drive. From a NON-persistent session, mount the drive and change the partition label from "persistence" to something else. From a NON-persistent session, mount the drive and delete or rename the "persistence.conf" file. If persistence is thus temprarily disabled, be careful not to modify the special files and folders on the persistence partition which hold the overlay data, or the next persistent session might be corrupted. Creating a dual-disk bootable ISO image with persistence ------------------------------------------------------------ Use the "Disks" utility to create a single-boot USB medium as usual. Create a second USB drive and prepare it with a persistence partition. Use "Disks" to select an (empty) USB disk, and format a single EXT4 partition labeled "persistence". Create a "persistence.conf" file as described above, containing the single line "/ union". To use the dual-disk method, boot from the desired ISO image medium. Before selecting "Live Boot" from the menu, insert the MATCHING persistence disk. Ensure that "persistence" is included in boot parameters, and boot. Be sure to leave the persistence disk in the system for the entire live session and properly shutdown the system when done.