Dev%2FAbout_Debian_Packaging.mw

__FORCETOC__
{{Header}}
{{#seo:
|description=Debian Packaging of {{project_name_long}} Design Documentation / Recommends vs Depends / --no-install-recommends / Files in Home Folder / Files in /etc/skel
}}
{{intro|
Debian Packaging of {{project_name_short}} Design Documentation / Recommends vs Depends / --no-install-recommends / Files in Home Folder / Files in /etc/skel
}}
= Purpose of Packaging =
Packaging serves several crucial purposes in software distribution, ensuring not only the smooth integration and functioning of software but also maintaining standards of security, legality, and consistency. Here's a breakdown of its primary objectives:

* '''Security hot fixes:''' Distributions can provide timely solutions when the original software creators are slow to respond or unavailable. This is vital since many developers are volunteers who may be preoccupied with other responsibilities.
* '''License compliance:''' Distribution maintainers (DMs) verify that the software adheres to freedom-respecting licenses, ensuring legal compliance and ethical use. See [[Avoid nonfreedom software]] for more details.
* '''No binary blobs:''' DMs ensure that the source package doesn't include pre-compiled binary files, promoting transparency and security.
* '''Source code completeness:''' This involves verifying whether the source code is complete and buildable, ensuring that the end-users receive a functional product.
* '''No embedded copies:''' Packages should not contain outdated or vulnerable third-party libraries as embedded copies. Refer to [https://wiki.debian.org/EmbeddedCopies Embedded Copies] for more information.
* '''Build reproduction:''' Adhering to [https://reproducible-builds.org/ Reproducible Builds] practices allows for the creation of binaries that can be independently verified, enhancing trust and security.
* '''Download mirroring:''' In case the original sources are temporarily unavailable, distribution mirrors ensure that the software can still be downloaded and used.
* '''Hardening:''' Distributions may implement additional security measures to enhance protection against threats. This could vary based on the individual DM or team, including:
** '''AppArmor:''' Implementing mandatory access control, like an AppArmor profile, especially if it's not maintained by the original software providers.
** '''Compilation options:''' Utilizing more secure compilation options, which can be independent of the packages and applied distribution-wide.
* '''Distribution integrations:''' Integration with the distribution's specific systems is crucial. For example, distributions using different initialization systems (like systemd, OpenRC, Runit, or SysVinit) require the DM to maintain appropriate init files for their system. For instance, Devuan, a non-systemd distribution, uses SysVinit and thus needs SysVinit scripts for all packages involving daemons (like sound servers, display servers, or internet servers).
* '''Linting:''' Package checkers are used to automatically identify issues and enforce quality and standards in the packaging process.

{{quotation
|quote=Lintian dissects Debian packages and reports bugs and policy violations. It contains automated checks for many aspects of Debian policy as well as some checks for common errors. Every Debian maintainer should check packages with this tool.
|context=[https://salsa.debian.org/lintian/lintian Lintian on Debian Salsa]
}}
{{quotation
|quote=Command-line tool and website running thousands of checks to analyze packages, suggest fixes for source, packaging, software, and spelling, useful across various distros and platforms, not just Debian.
|context=fosdem presentation, [https://archive.fosdem.org/2021/schedule/event/mariadb_distros/attachments/slides/4381/export/events/attachments/mariadb_distros/slides/4381/FOSDEM_2021_MariaDB_post_release_QA_in_Debian_and_Ubuntu_by_Otto_Kekalainen.pdf MariaDB post-release quality assurance in Debian and Ubuntu]
}}
** Lintian has [https://salsa.debian.org/lintian/lintian/-/tree/master/tags a plethora of checks]
* '''Architecture support:''' Upstream often provides packages and/or source code for a limited number of architectures (or Intel / AMD64 only). Distributions often build packages for multiple architectures. To learn more how Debian does that, see [https://buildd.debian.org/ Debian Buildd - Debian Package Auto-Building].
* '''Continuous Integration:''' For example, [https://ci.debian.net/ debci - Debian Continuous Integration].
* '''Quality Assurance:''' For example, [https://piuparts.debian.org/ piuparts]

{{quotation
|quote=piuparts is a tool for testing that .deb packages can be installed, upgraded, and removed without problems. piuparts is short for "package installation, upgrading and removal testing suite"
}}

{{quotation
|quote=It does this by creating a minimal Debian installation in a chroot, and installing, upgrading, and removing packages in that environment, and comparing the state of the directory tree before and after. piuparts reports any files that have been added, removed, or modified during this process. piuparts is meant as a quality assurance tool for people who create .deb packages to test them before they upload them to the Debian package archive.
}}
* '''Package Uninstallation Support:'''
** Want to remove something installed from upstream using <code>sudo make install</code>? Good luck. That's usually by upstream considered the job by the distribution packaging.
** Removing software installed directly from upstream sources can be a challenge. This is often because upstream developers typically consider the management of installation and uninstallation processes, including clean removal of installed files, to be the responsibility of distribution packaging rather than their own. Using commands like <code>sudo make install</code> often doesn't provide an easy reversal option, highlighting the importance of distribution packages in facilitating straightforward uninstallation.

= Lintian =
All packages produce result in <code>lintian</code> exit code <code>0</code> and no <code>lintian</code> output. All <code>lintian</code> warnings are either:

* fixed
* suppressed (<code>--suppress-tags</code>
** hardcoded in package <code>genmkfile</code>
** file <code>make-helper-overrides.bsh</code>
** <code>debian/make-helper-overrides.bsh</code>)
* have override configurations
** file: <code>debian/source/lintian-overrides</code> or more often in:
** file: <code>/usr/share/lintian/overrides/package-name</code>

When running <code>genmkfile deb-pkg</code>, <code>lintian</code> should be automatically executed, if installed.

<pre>
/usr/share/genmkfile/make-helper-one.bsh: INFO: You can find your deb file here: /home/user/derivative-binary/genmkfile-packages-result/privleap_1.3-1_all.deb
/usr/share/genmkfile/make-helper-one.bsh: INFO: make_use_lintian='' - Autodetecting if lintian is installed...
/usr/share/genmkfile/make-helper-one.bsh: INFO: lintian auto detected, using it...
lintian           --suppress-tags missing-tests-control          --suppress-tags systemd-service-file-missing-documentation-key          --suppress-tags orig-tarball-missing-upstream-signature          --suppress-tags package-supports-alternative-init-but-no-init.d-script          --suppress-tags no-manual-page          --quiet          --pedantic          --info          --display-info            "/home/user/derivative-binary/genmkfile-packages-result/privleap_1.3-1_amd64.changes"
/usr/share/genmkfile/make-helper-one.bsh: INFO: lintian exit code: 0
/usr/share/genmkfile/make-helper-one.bsh: INFO: lintian output:
</pre>

= Analyzing Dependency Chains =
== reverse-depends ==
{{Box|text=
'''1.''' Modify <code>/etc/dpkg/origins/default</code>.

{{CodeSelect|code=
sudo unlink /etc/dpkg/origins/default
}}
{{CodeSelect|code=
sudo ln -s /etc/dpkg/origins/debian /etc/dpkg/origins/default
}}

'''2.''' Download the source package.

{{CodeSelect|code=
apt source package
}}

'''3.''' Undo afterwards to prevent unexpected issues.

{{CodeSelect|code=
sudo unlink /etc/dpkg/origins/default
}}
{{CodeSelect|code=
sudo ln -s /etc/dpkg/origins/whonix /etc/dpkg/origins/default
}}
}}

{{Install Package|package=
ubuntu-dev-tools
}}

{{CodeSelect|code=
reverse-depends evolution-data-server
}}

== meta packages ==
* https://github.com/{{project_name_short}}/kicksecure-meta-packages/blob/master/debian/control

= Recommends vs Depends =
This is about <code>Recommends:</code> vs <code>Depends:</code> in context of <code>debian/control</code>.

<code>debian/control</code>: There are separate [https://github.com/{{project_name_short}}/{{project_name_short}}-meta-packages/blob/master/debian/control meta packages] for dependencies and recommended packages. For example:

* <code>{{project_name_short_lowercase}}-xfce</code>
* <code>{{project_name_short_lowercase}}-cli</code>

The reason for this is, because if we used the <code>Recommends:</code> field for {{project_name_short}} meta packages (those who pull the required Debian upstream packages for creating {{project_name_short}}), we could not install them using <code>apt</code> with <code>--install-recommends</code>, which is <code>apt</code>'s default option, because that would also install packages recommend by any dependency we install.

On the other hand, if we installed using apt <code>--no-install-recommends</code>, the packages {{project_name_short}} meta packages recommends, will not get installed.

Therefore splitting them into packages suffixed <code>*-dependencies</code> or <code>*-recommended</code> which both use <code>Depends:</code> and installing them using <code>--no-install-recommends</code> appeared to be the only solution.

Otherwise it would install packages such as <code>virtuoso</code>, <code>soprano</code> and <code>vlc</code>, which are not useful '''in context of''' {{project_name_gateway_long}}.

The <code>Recommends:</code> and <code>Suggests:</code> field is still being used but this is mostly useful for one package advertising related packages users using <code>apt-cache show package-name</code> and [[Packages for Debian Hosts]].

See also:

* [[Install_Software#--no-install-recommends|<code>--no-install-recommends</code> user documentation]]
* [[Debian_Packages#Technical_Information]].

= Files in Home Folder =
Packages placing files <code>/home</code> folder must use the correct mechanism for doing so. Using defined mechanisms (see below) might be appropriate. Directly writing <code>/home</code> is prohibited for reasons listed below.

* <code>/home</code> is for users. Not for distribution contributors.
* Leads to a [[Configuration_Files#dpkg_interactive_conflict_resolution_dialog|dpkg interactive conflict resolution dialog]] when package file is updated, in case file gets modified by the user or a program in the home folder (which is often very likely), which is a usability issue, which we try to avoid.
* <code>serious</code> <code>lintian</code> error [https://web.archive.org/web/20200927070851/https://lintian.debian.org/tags/dir-or-file-in-home.html <code>dir-or-file-in-home</code>].
** Makes the package unfit for inclusion into <code>packages.debian.org</code> (very long term goal) (or other package repositories).
** Looks amateurish in the eyes of Debian packagers.
* For which user? User <code>user</code> only? Inconsistent for multi user use cases. Hard to combine with future [[Dev/user-sysmaint-split|user-sysmaint-split (Role-Based Boot Modes)]] with <code>user</code>, user <code>sysmaint</code>.
* Doesn't work / inconsistent in [https://www.qubes-os.org Qubes] Template. Since packages are usually upgraded in Templates, the change never propagates to the home folder of the App Qube since it has an independently persistent home folder.
** Special code would be required to handle these cases. Such as a script that runs after boot but in Template that are not Disposable Template only. Example:
*** https://github.com/{{project_name_short}}/tb-updater/blob/master/usr/lib/systemd/system/tb-updater-first-boot.service
*** More code to carry around, understand interactions, bugfix, maintain, explain, audit.
* In most cases there are more suitable mechanisms to reach the implementation goal than writing into the user's home folder.
** [[#Files in /etc/skel|Files in /etc/skel]]
** If not, the lack of such mechanisms should be discussed with / requested from upstream.

= Files in /etc/skel =
* Files in <code>/etc/skel</code> are not as bad as files in [[#Files in Home Folder|<code>/home</code> folder]].
** Works for any user.
* Inconsistencies. Not deployed through <code>/etc/skel</code> mechanism if file is added to a package after a user account was created. I.e. users who upgraded will miss that file.
** Needs special code to handle such cases.
* If the file from <code>/etc/skel</code> is in the user's home folder, it's hard to update it. Updating the file in <code>/etc/skel</code> won't effect the user's version of the file in the user's home folder.
** Needs special code to handle such cases.

= Modifying Default Configuration of Third Party Packages =

Taking systemd as an example. Suppose <code>/lib/systemd/system/systemd-random-seed.service</code> modifications are desired.

File <code>/lib/systemd/system/systemd-random-seed.service</code> is owned by a third party package <code>systemd</code>.

{{CodeSelect|code=
dpkg -S /lib/systemd/system/systemd-random-seed.service
}}

<pre>
systemd: /lib/systemd/system/systemd-random-seed.service
</pre>

* Editing <code>/lib/systemd/system/systemd-random-seed.service</code>.
** Is the worst (changes get lost on systemd package gets upgraded) (there is a solution using <code>dpkg-divert</code> encapsulated in a more sane by using <code>config-package-dev</code> <code>displace</code> and we use it in {{project_name_short}} source code where it could not be avoided but if avoidable it is best avoided if possible somehow).
* Shipping <code>/etc/systemd/system/systemd-random-seed.service</code>
** would be better but still not great (this is for local administrator, not linux distribution)
* <code>/lib/systemd/system/systemd-random-seed.service.d</code> folder and a file such as <code>/lib/systemd/system/systemd-random-seed.service.d/30_something.conf</code>
** is an ideal solution for linux distributions such as {{project_name_short}} since it does not takeover files by other packages and does not take away something from the local system administrator or user.

Editing a file owned by a different packages residing in <code>/etc</code> would be even worse. Suppose for example file <code>/etc/hdparm.conf</code> was modified by a package by a derivative Linux distribution. Next time the base distribution (Debian) updates the file, this would lead to an {{Code2|dpkg interactive conflict resolution dialog}}.

<pre>
Configuration file `/etc/hdparm.conf'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : background this process to examine the situation
 The default action is to keep your current version.
*** interfaces (Y/I/N/O/D/Z) [default=N] ? N
</pre>

This is because the package manager (APT) cannot merge upstream (Debian) file changes with local file changes.

Such prompts confusing for users as users do not know what to answer. These result in support requests. Whatever the user chooses is non-ideal. If the user chooses the package maintainer's (Debian's) version, the user will loose settings modifications by the derivative. If the user keeps the file, the user will miss out on updates by the package maintainer. Merging the file is a skill that only the minority of users possesses.

This would not be an issue if the derivative distribution would fork, rebuild, upload the package that owns that file. In this example if the derivative distribution would upload package <code>hdparm</code> then there would be no issue updating <code>/etc/hdparm.conf</code> because the derivative would not "compete" with the base distribution. This however is not how it is done for derivatives such as Kicksecure, Whonix or Qubes. If just a small configuration change is required, the huge extra effort of a complete fork of the package is avoided to safe time. Only a drop-in configuration snippet gets shipped instead.

= use /usr instead of /etc =
* <code>/usr</code>: Is for distributions, packages.
* <code>/etc</code>: Is for system administrators.

If possible, avoid <code>/etc</code> and use <code>/usr</code> instead.

Why distributions are still making use of <code>/etc</code>? While it's impossible for all distributions, but the most likely reason is legacy. This is for historic reasons. <code>systemd</code>, which continues to shape the direction which Linux distributions develop, many Linux distributions and upstream projects agree with this direction of moving to <code>/usr</code>. 

{{quotation
|quote=Note that while <code>systemd</code> itself explicitly supports booting up with entirely unpopulated images (<code>/usr/</code> being the only required directory to be populated) distributions might not be there yet: depending on your distribution further, manual work might be required to make this scenario work.
|context=[https://systemd.io/BUILDING_IMAGES/ Building Images Safely]
}}

{{quotation
|quote=While a lot of work has gone into ensuring systemd systems can safely boot with unpopulated /etc/ trees,
|context=[https://systemd.io/BUILDING_IMAGES/ Building Images Safely]
}}

[https://uapi-group.org/ The Linux Userspace API Group] quote ticket [https://github.com/uapi-group/specifications/issues/76 Tracking upstream projects that do not support hermetic-usr for configuration] lists Linux software packages that still require configuration in <code>/etc</code> that do not support the <code>/usr</code> folder for configuration yet.

{{quotation
|quote=Hermetic <code>/usr/</code>

[...]

How can you help?

[...]

At this point some distributions (such as Fedora) are (probably more by accident than on purpose) already mostly hermetic in <code>/usr/</code>, at least for the most basic parts of the OS. However, this is not complete: many daemons require to have specific resources set up in <code>/var/</code> or <code>/etc/</code> before they can work, and the relevant packages do not carry <code>systemd-tmpfiles</code> descriptions that add them if missing. So there are two ways you could help here: politically, it would be highly relevant to convince distributions that an OS that is hermetic in <code>/usr/</code> is highly desirable and it's a worthy goal for packagers to get there. More specifically, it would be desirable if RPM/dpkg packages would ship with enough <code>systemd-tmpfiles</code> information so that configuration files the packages strictly need for operation are symlinked (or copied) from <code>/usr/share/factory/</code> if they are missing (even better of course would be if packages from their upstream sources on would just work with an empty <code>/etc/</code> and <code>/var/</code>, and create themselves what they need and default to good defaults in absence of configuration files).
|context=systemd blog post [https://0pointer.net/blog/fitting-everything-together.html Fitting Everything Together]
}}

Marek (Qubes OS lead developer) also agrees with this.

{{quotation
|quote=, /etc is for user changes, packages should avoid modifying config files there of other packages if possible.
|context=[https://github.com/QubesOS/qubes-issues/issues/8832#issuecomment-1881139726 Qubes issue tracker]
}}

Patrick (Kicksecure lead developer) also agrees with this.

= config-package-dev =
* https://packages.debian.org/config-package-dev
* https://debathena.mit.edu/config-packages/

Debian package <code>xfce4-session</code> owns file <code>/etc/xdg/xfce4/xfconf/xfce-perchannel-xml/xfce4-session.xml</code>. Therefore package <code>xfce-desktop-config-dist</code> cannot directly overwrite it.

<pre>
Preparing to unpack .../xfce-desktop-config-dist_1.4-1_all.deb ...
Unpacking xfce-desktop-config-dist (3:1.4-1) ...
dpkg: error processing archive /mnt/initialdeb/pool/main/w/xfce-desktop-config-dist/xfce-desktop-config-dist_1.4-1_all.deb (--unpack):
 trying to overwrite '\''/etc/xdg/xfce4/xfconf/xfce-perchannel-xml/xfce4-session.xml'\'', which is also in package xfce4-session 4.12.1-6
Errors were encountered while processing:
 /mnt/initialdeb/pool/main/w/xfce-desktop-config-dist/xfce-desktop-config-dist_1.4-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)       '
+ apt_get_exit_code=100
</pre>

config-package-dev can help with such situations.

Examples on how to use config-package-dev displace:

* https://github.com/{{project_name_short}}/desktop-config-dist/commit/0e9daa97e9f9e70120c969aa9c9d52cace46971a
* https://github.com/{{project_name_short}}/security-misc/commit/93c08210545dd77b608515351154bcc16c8464b4

Ideally avoided as per above.

* upstream bug report [https://github.com/sipb/config-package-dev/issues/7 dpkg-divert: warning: diverting file '/etc/issue' from an Essential package with rename is dangerous, use --no-rename]

= Desktop Environment (Xfce) Settings Changes =
Testing modified desktop environment (Xfce) settings files in <code>/etc/skel</code> or <code>/home/user</code> is cumbersome.

When Xfce is shutdown, it might undo the manual or package modified settings in user home folder <code>/home/user</code> as part of Xfce "save settings" procedure on shutdown. Therefore after reboot these settings might be reverted and could not be actually tested.

A script which is supposed to be run from a virtual console to simplify the steps:

[https://github.com/{{project_name_short}}/helper-scripts/blob/master/usr/libexec/helper-scripts/desktop-background-skel-test <code>/usr/libexec/helper-scripts/desktop-background-skel-test</code>]

= {{project_name_short}} / {{whonix}} Package Design =
* whonix-base-files: Whonix-Host, Whonix-Gateway, Whonix-Workstation
* whonix-libvirt: Whonix-Host only
* anon-gw-base-files: Whonix-Gateway only
* anon-ws-base-files: Whonix-Workstation only
* kicksecure-base-files: Kicksecure-VMs and Kicksecure-Hosts
* vm-config-dist: Kicksecure-VMs and Whonix-VMs

= Package File Migration =
Example.

Migrating a file in a package such as /usr/sbin/pwchange from one package to another.

Migrating a file in a package such as /usr/sbin/pwchange from usability-misc to helper-scripts.

<pre>
helper-scripts debian/changelog:

    helper-scripts (3:28.2-1) unstable; urgency=medium
    ...

usability-misc debian/changelog:

    usability-misc (3:26.4-1) unstable; urgency=medium
    ....

helper-scripts debian/control:

    Package: helper-scripts
    ...
    Breaks: usability-misc (<< 3:26.4-1)
    Replaces: usability-misc (<< 3:26.4-1)
    ...

usability-misc debian/control:

    Package: usability-misc
    ...
    Depends: ... helper-scripts (>= 3:28.2-1) ...
    ...
</pre>

= See Also =
* [[Dev/Versioning Format Conventions|Versioning Format Conventions]]
* [[Dev/Build_Documentation|Build Documentation]]
* Build Documentation to build any package by {{project_name_short}} such as [[Dev/Build Documentation/security-misc|security-misc]]
* [[Dev/Build_Documentation/generic-package]]

{{Footer}}

[[Category:Design]]