\chapter{Philosophy}

When making something, one always faces choices on how to do it,
weighing the implications for and against the different design
choices.

\section{Good versus bad design}

One can argue whether good and bad is relevant when it comes to
design.  To a certain degree and in a certain context it is.  A
quick hacked-together solution is not good design if you want
something that lasts and is easy to for others to extend.  As a
software distribution system will be used by and worked on by many
different people.  Those will of course wary in both roles
(installing and maintaining software, building packages, changing the
distribution system itself) as well as in skill.  This seems obvious,
but when one looks at the different systems out there, it is clear
that many people wrote code first and thought about the problem later,
rather than the other way around.

It is also important that the different design choices support each
other.  If not, the user will be confused and the software may be slow
and full of errors.  The errors will be in the user interface (because
the metaphors used are not consistent) as well as in functionality.
Some areas will be covered by multiple parts of the program while
other areas are not covered at all.

\section{Who is the user}

Users are different and have different goals, usually dictated by
their role.  A system administrator is usually not interested in
working on the packaging system itself.  He is more interested in
making or installing pre-made packages.  To a packager, installing
pre-made packages is of little interest, he will be interested in
making good packages with as little hassle as possible.  A new user
wants the system to be easy to get into while an experienced user will
value work-saving features.  Note that there isn't a necessary
conflict between those goals.

\subsection{Operations}

The number of operations a software distribution tool must support is
surprisingly small.  Those operations aren't atomic, but they are
operations needed in a full-blown software distribution system.  The
different options and operations listed here are mostly intended as
thought-starters rather than being a comprehensive list of all the
choices available to the designer of a software distribution system.

\begin{figure}[h]
  \centering
  \includegraphics[height=8em]{package_lifecycle}
  \caption{The life cycle of a package}
  \label{fig:package_lifecycle}
\end{figure}


\begin{itemize}
  \item {\em Production}.
    Without the ability to produce packages, the rest of the system is
    unusable.  When deciding what the production step should look
    like, there are some options available:
    \begin{itemize}
    \item {\em Pure (pristine) source or patched source?}  The trend is
      certainly towards having pristine tarballs from the distribution
      source, as this makes it easy to check the integrity of those
      using tools such as md5sum or sha1sum and using separate patches
      on top of this.
    \item {\em Form of build scripts.} Build scripts can be abstract,
      saying ``put this file there''.  Abstract build scripts don't
      have commands in them, their function is decided by their name
      or position in the build tree.  Non-abstract, or real scripts
      can also be used.  Examples are shell scripts or makefiles.
      Those list the commands to be executed.
    \end{itemize}
  \item {\em Distribution} can be in source or binary forms, it can be over a
    network connection or on CD or other physical media.  The former
    is decided by the software distribution system, the latter is
    usually not.  There are wide support for both source- and binary
    based distributions and they give different abilities:
    
    \begin{itemize}

    \item Source-based distribution means you can turn off support you
      don't need, which can decrease the size needed by the system.
      In most modern systems, this will not make any real difference,
      but some people think it's very much worth it.

    \item Binary distribution has the advantage of much easier
      reproducibility (if you install the same set of binary packages,
      you will always end up with the same setup.)
    \end{itemize}

    Note that source-based or binary-based is orthogonal to free
    software/open source, a source based distribution can ship
    pre-compiled objects which are linked together depending on the
    configuration.  (An example of this is when booting the Solaris
    kernel, it relinks itself.  The rest of the Solaris operating
    system is distributed as binary-only.)

  \item {\em Initial installation} of packages.  One of the most
    important choices when it comes to how to install packages, is the
    choice of complexity.  The simplest solution is to have a cpio or
    tarball which is just extracted onto the system.  It becomes a bit
    more complex if you add pre- and post-installation and -removal
    scripts.  Even more complex if you add classes of files, so the
    system administrator can say ``I don't want documentation
    installed'', or ``Those architecture-independent files can be
    accessed via NFS from that server''.

  \item {\em Maintenance} including querying capabilities and
    upgrading of configuration files.  In some cases, another setting
    is needed or the syntax has changed.  In those cases, you want to
    preserve the local modification while including the upgrade.  It's
    not always possible to automatically upgrade the configuration.
    Then the administrator must be told of the problem so he can
    resolve it by hand.

  \item {\em Configuration of the software distribution system}.  All
    systems have a number of built-in default values, here called
    implicit configuration.  In addition, one usually has some kind of
    configuration file or tree where those default values are
    overridden.  Striking a balance on what should be configurable and
    what shouldn't be is important, both for speed, stability and user
    friendliness. 

    Not only the amount of configuration, but also the form of
    configuration is important.  It will be explored a little bit more
    in the next section, as part of the user interface.  It is
    important to note that there is a grey area between configuration
    and customisation of the distribution system, if the configuration
    language has the needed expressive power.

    Too much implicit configuration can be a problem, since the
    administrator then doesn't understand how (or has to learn how)
    his configuration interoperates with the default configuration.
    On the other side, the administrator is usually interested in
    turning the minimum number of knobs.

\end{itemize}

\subsection{User experience paradigms}

Designing user interfaces is a very hard task.  An interface that is
simple and easy-to-use for one person might be useless for another
person, since the latter will not have the same connotations connected
to the metaphors in use.  This is especially true for configuration
languages.  Some can be in the form of production rules (like
Makefiles).  This is a very beautiful and very simple concept, but it
can take a while to get used to it.  Other languages will be a simple
key~=~value form.

Many packages will need to have support scripts of some sort, for
deciding on values to put into configuration files, call other
utilities to notify them of the installed package and integrate the
package properly into the rest of the system.  Most packagers aren't
really programmers, they just write those support scripts as a
by-product of the packaging.  This means they need a fairly simple way
of programming.  The simplest is the so-called ``standard execution
point'' or procedural programming.  For bigger and more complex
packages, one needs support for structured programming.  

\section{Operational paradigms}
\label{ph_operational_paradigms}
There are several ways to go about in order to reach a certain goal.
Push isn't better than pull, or the other way around.  They are
different and give you different abilities, strengths, weaknesses, and
options on how to design the rest of the system.  The goal is to
choose a good set whose sum is as close as possible to where one wants
to end.

Some important paradigms are:

\begin{enumerate}
  \item {\em Push/pull}

    \begin{figure}[h]
      \centering
      \includegraphics[height=8em]{push}
      \caption{Pushing}
    \end{figure}

    Push means a central server knows the current state of each of the
    clients and pushes out the needed information to the clients.
    One gets fewer coordination points and just one machine to do
    backups of, but at the same time, the system becomes vulnerable
    with a single point of failure.  

    The server not only decides what changes should be pushed, but
    also at what time they should be pushed, which could be an
    inconvenient time for the client.  In practice, the push server
    also needs to be able to log in as the administrative user on each
    of the clients, making push a possible security issue, even more
    so if the clients are in separate administrative domains.

    Multi-level hierarchies are also harder to implement using push,
    as the central host will have a hard time knowing the state of all
    the clients due to having the state pushed through (possibly
    several) intermediaries.

    \begin{figure}[h]
      \centering
      \includegraphics[height=8em]{pull}
      \caption{Pulling}
    \end{figure}

    Pull means the clients pull down whatever information they need
    from a server. The clients know their own state thereby removing
    the need for a central database.  Among the advantages of pulling
    are client control -- the client can pull when it's convenient for
    it to update.   It can also choose what it wants to pull.  This
    gives the clients control over the process rather than having a
    central policy dictating how the distribution shall be done.

    A pull-based system is harder to get an overview of, as the state
    information is spread over all the hosts.  It is also harder to
    know if a host is down or not updating properly, as it will just
    stop doing what it is supposed to do.

    Most systems will not be one or the other, even systems like
    rdist\cite{rdist-site}, which are nominally push, will look to see
    if they actually need to push a newer version of the file, rather
    than storing all the state on the central server.

  \item {\em What to push/pull: Set of operations or wanted state?}

    One can transfer either a set of operations or the wanted state
    one wants the system to get to.

    Pushing (or pulling) a set of operations is a lot simpler, both on
    the client which just has to execute the operations and on the one
    deciding what operations to send.  However, when pushing
    operations, a brute-force approach is often chosen, so instead of
    doing the minimum amount of work, a lot of runtime computer power is
    wasted in order to save human work.

    Pushing state to the client means more work goes into figuring out
    how to get from the current state to the wanted state, but less
    work goes into actually getting there.  This is considered a lot
    more elegant than pushing operations, but is harder to do right.

  \item {\em Autonomous units, immune systems}
    
    Viewing the computer as a ``living system'' where the software
    configuration management system works like an immune system is a
    fairly recent trend.  This is a very interesting idea, but is
    complex to design and develop as well as debug.  Mark Burgess, of
    the Oslo University College is spearheading the effort, with his
    cfengine tool\cite{cfengine-site}.  This design is most often
    combined with the ``wanted state'' approach.

  \item {\em Tool structure} 

    Different operating systems have different styles in which most of
    the tools are made.  Some have large tools which can do
    everything, some have small tools which only do one thing, but do
    it really well.  The tools interoperate through different methods:
    pipes and object invocation (COM, CORBA) are among the options.

    \bigskip
      \begin{tabular}[h]{|l|l|l|}
        \hline
        Operating system & Tool size & Interoperation method \\
        \hline
        \unix & Small & pipes \\
        \label{tool_structure_vms} \vms  & Large & N/A\footnotemark \\
        Windows & Large & object invocation\\
        \hline
      \end{tabular}
    \bigskip
      \footnotetext{In \vms, a switch would be added, or 
        one would use temporary files. }

    \item {\em Tool size}

      Smaller tool size facilitates debugging and lower memory demands
      at the cost of more processes and often speed.

      Bigger tools can be better integrated, will often be faster in
      total, due to said integration but will consume more memory, as
      all the functions are loaded even if one just uses a subset.

    \item {\em Interoperation method}

      Pipes are a very simple concept, but they seem to scale
      amazingly well.  They are flexible, since one can add another
      process in the middle of a pipeline.  Pipes are usually combined
      with small tools, but they can work fine with large tools as
      well.  Among the disadvantages, we have brittleness (it's hard
      to detect if one of the steps in the pipeline fails) and limited
      expressibility as flow control is fairly limited.

      Object invocation seems better for gluing applications together
      and embedding applications in other applications rather than
      having applications work together.  It makes complex
      interactions between different modules fairly painless at the
      cost of higher general complexity.


%  \item {\em policies vs explicit statements}
% possible fixme: write up on the above one    

\end{enumerate}


\section{Programming interfaces}

Since no tool is complete by itself, most tools will be used together
with other tools.  It will be embedded and called from other tools and
extended with new functionality.  Some extensions are best done by
plugins or extending the program, while others are best done wrapping
the software in another program.  This mean the different ways should
all be supported, not just one.

Some of the programming interfaces available to extend a system are:

\begin{itemize}
  \item {\em Plugins}

    Support for plugins means custom code can decide how to handle
    items such as file types, like the example on ``don't install
    documentation'' can be changed into ``put all documentation into
    this other (web-accessible) location''.

    It can also be used for adding support for other distribution
    forms, other formats and so on.  Plugins are usually used for
    extending the abilities of a program rather than changing how the
    internal work flows.

  \item {\em Hooks}

    Hooks are scripts, programs or library functions that are
    activated at predefined points during the execution of another
    function or program.

    Examples of this would be a package wanting to know when
    another package is installed, since it would then need to
    reconfigure itself (imagine an emacs add-on wanting notification
    about a new version of emacs being installed) or SELinux (Security
    Enhanced Linux) wanting to adjust the security contexts of newly
    installed files.

    Also, the ability to run a program once when a software package
    has completed installing would be a hook.  An example would be
    regenerating the menu files for the installed window managers when
    a new program is installed.

  \item {\em Application Programming Interfaces}

    If the software distribution system is used as a part of another
    larger system, it's most useful to be able to drive the
    distribution system programmatically.  Well-defined APIs make
    error reporting and general feedback (such as progress meters)
    easy.  Providing a set of APIs also means the system has a set of
    well-defined operations which are exposed to the rest of the world
    and the inner workings properly hidden inside the application.

\end{itemize}

\section{Summary}

There are quite a number of design choices which affect the basics of
the system quite heavily.  If those are ill-chosen with regards to the
wanted behaviour of the system, the system will never be nice, smooth
and well-working, but rather inconsistent and surprising on the user
with missing functionality.

It is also important to note that some of the choices goes well with
other choices while other are bad fits, so the set of options listed
here is neither complete nor completely orthogonal.

As it goes for tools, ``one size fits all'' doesn't fit here either.
Perfect tools are just a dream, but a combination of different tools
fitting well together comes fairly close.  Even if somebody made a
perfect tool today, it would not fit tomorrow's requirements.
Extensibility is therefore a keyword, and it needs to be built in from
the start.

