RMF {#rmf}
===

The IMP.rmf module acts as a link between RMF files and %IMP data structures.
Currently, three types of data are linked:

- IMP::Restraint
- IMP::atom::Hierarchy
- IMP::display::Geometry

For each type of data, there is a set of functions declared in `foo_io.h`
and defined in `foo_io.cpp`. These functions support four major operations

- addition of new data to the RMF file: this creates a hierarchy of nodes
  in the RMF and stores a link between those nodes and the corresponding
  IMP::Objects
- creation of %IMP structures from the RMF: %IMP data structures are created
  based on corresponding data contained in the RMF file
- link existing %IMP structures to the RMF: existing %IMP data structures
  are linked to data stored in the RMF. The two must correspond exactly
  (eg the RMF must have been created by adding those %IMP data structures
  to the file).
- saving a frame to a file: a new frame is added to the RMF file and all
  linked data is used to write data to that frame.

The links between %IMP data structures are done via IMP::rmf::LoadLink and
IMP::rmf::SaveLink. These are data structures associated with the RMF file
(the RMF library provides a mechanism for this) that store persistent state
that is available as long as the file is open. There is one load and one
save link class per type of data.

## Hierarchy ##
atom::Hierarchy data is stored as RMF::REPRESENTATION nodes in the file.

There is exactly one node in the RMF file per IMP::Particle in the hierarchy.
The link classes are implemented in terms of a bunch of helper classes,
each of whichs handle a particular type of data.

The main tricky part is handling rigid bodies. In particular, the particles
defining the rigid body are not always part of the %IMP hierarchy, and
older RMF files (where rigid bodies were written differently) must also be
handled.

Rigid bodies that are not part of the IMP::atom::Hierarchy are handled by

- at each node in the hierarchy, checking if all core::XYZ particles in the
  subtree are part of the same rigid body
- if so, pretend that the current particle is the actual rigid body.
- since this can result in one %IMP rigid body having many corresponding
  nodes in the RMF, a tag (an id integer) is added to such rigid bodies
  so they can be combined when creating from the file

When writing coordinates of things that are rigid bodies, the internal
coordinates are written to the RMF rather than the global coordinates. This
makes the files more compact. It works since the rigid bodies are written as
RMF::decorator::ReferenceFrame nodes in the file, which effectively provides
a transformation to the coordinates of things underneath it.

## Restraints ##
Restraints are stored as RMF::FEATURE nodes in the file.

Restraints are more or less write only, in contrast to the other types. There
are create functions, but since the RMF doesn't contain enough data to
actually create the correct %IMP type, these just create dummy restraints
whose score is simply the value stored in the file (and which don't have any
derivatives). As a result, it is not clear that they can be used for much
beyond verifying scores and some simple analysis.

Each IMP::Restraint added to the RMF is written as follows:

- a RMF node is created for the restraint itself, containing one score per frame
- IMP::Restraint::create_current_decomposition() is called on the restraint.
  If this decomposition contains more than one restraint, then child nodes
  are added for each restraint and they are stored recursively.
- all leaves have the (static) list of RMF nodes that they involve written.

Mapping between the sub-restraints created by create_current_decomposition()
and RMF nodes is slightly non-trivial. It is done based on the set up input
particles for each restraint returned by create_current_decomposition().
For each such restraint, the link first checks in a map if there is already
a node with that set of inputs, if so, it reuses that. If not, it creates
a new one.

## Geometry ##
Geometry is stored as RMF::SHAPE nodes in the file.

The functionality provided in IMP::display::Geometry doesn't map entirely
naturally on to RMF's geometry support. On the %IMP side, things are
implemented in terms of an ephemeral decomposition of complex geometric
objects into simple ones (eg, a bounding box can be decomposed into edges,
but each time you ask for the decomposition, you will get a different set
of IMP::Objects). The RMF side expects a hierarchy whose structure stays
constant across frames. As a result, each type of geometry object has to
be special cased in IMP.rmf. Currently there is support for

- segments
- balls
- bounding boxes
- cylinders
- points

## Considerations ##
For each piece of data stored in the RMF, a decision has to be made whether
to store it once per file (using the RMF set_static_foo() methods), once
per frame (using the RMF set_frame_foo() methods), or automatically
(using RMF set_foo() methods). In IMP.rmf, in general things are stored
once per file with the exception of coordinates/transformations of things
that are not rigid parts of rigid bodies, which are stored once per frame.
This could be changed to using automatic for almost everything, but that is

- a bit slower as it has to check each time if the value is different
  than what is already stored
- not something that other programs expect for things like residue index

In particular, it might make sense for radii to become automatic.

One must be very careful not to store any RMF::NodeHandle or RMF::FileHandle
objects in the adaptors as these will keep the RMF file open indefinitely
(since the handles will keep the file alive and the file keeps the adaptor
alive and the adaptor keeps the handle alive).
