################################################################

$RCSfile: README.txt,v $

Author: Craeg Strong

$Date: 2003/04/21 01:52:42 $

################################################################


Contents

  1. Description

  2. Advantages, Disadvantages, and Alternative Products

  3. External Dependencies

  4. CVS Sandbox Registry

  5. Which Sandbox?

  6. How are files served up from sandboxes?

  7. Development Workflow

  8. Creating Instances Programmatically

  9. Contributions

  10. Schema Migration

  11. Notes

  12. Unit Testing on Win32
 
Description

  The CVSFile package is a Zope product.  CVSFile allows one to use
  CVS as a version control system for Zope content.  The developer
  creates a CVSFile, pointing to a file in the developer's CVS Sandbox
  on the Zope server.  Other Zope objects are able to access the
  content, but it is actually stored on the disk, rather than in the
  ZODB.  CVSFile enables the user to modify the file and do simple CVS
  commands.  CVSFile inherits from
  "ExternalFile":http://www.zope.org/Members/arielpartners/ExternalFile,
  a Zope product that behaves like a standard Zope object like File,
  Page Template,or DTMLMethod, but points to external content in the
  filesystem.  Please read the ExternalFile Product README before
  proceeding.

  The reason we are creating this new product is to support our usage
  of CVS for revision control.  We would like to be able to continue
  to store website content within CVS as we did in our pre-Zope days,
  while at the same time utilize the full power of Zope.

  With CVSFile, every developer can use the same, single Zope server
  instance **and yet they each see their own versions of files**.
  This is a crucially important point and a significant advantage for
  CVSFile.  It is done through browser cookies.  See below for a more
  detailed explanation.

  In this way, CVS is used for what it is best at: revision control,
  support for parallel development, branching, merging,
  etc. Similarly, Zope is used for what it is best at: support for
  acquisition, editing through the web, DTML, integrated security,
  undo, etc.

  Of course, CVSFile does not store everything in CVS.  The central
  idea behind CVSFile is to store files in CVS and store metadata in
  ZODB.  It is therefore most useful for those who already _have_
  significant content stored in CVS, but want to use Zope as a
  publishing environment.  You might even have multiple CVSFile
  instances in the ZODB all pointing to the same file in a CVS
  sandbox, but with different Zope metadata (properties, etc.).

  Note: CVSFile is not really intended for use in a production
  setting, (although we happen to use it that way :-).  Larger
  organizations tend to want tighter controls in production settings;
  for them, highly granular versioning and collaborative development
  are more important during the development and testing phases of a
  project.  In such organizations, in order to move into production,
  the contents of a website should go through a release process, part
  of which usually involves "freezing" or "snapshotting" the content.
  We hope to enhance CVSFile to better support such a "macro"
  development process in future.  For additional discussion, refer to
  the "Development Workflow" section.

  There are two ways to create CVSFiles from the Zope Management
  Interface (ZMI): one at a time or in a batch.  To use the batch
  option, simply select "CVS File Batch" from the standard Zope 'Add'
  menu.  It provides a way to automatically add an instance of CVSFile
  for every file within a directory.  Because CVSFile instances are
  merely links or aliases, they can be deleted without affecting the
  underlying file.  Therefore, an easy way to create CVSFile instances
  is to create them a batch at a time and then delete the unwanted
  instances (for example, those pointing to junk files, "tilde files",
  etc.).

  CVSFile is a subclass of ExternalFile, therefore it inherits all of
  its behaviors.  **Please refer to the ExternalFile README file for a
  important additional information**.

Advantages, Disadvantages, and Alternative Products

  Advantages: Files stored in CVS are directly usable by other tools
  besides Zope.  Zope stores the Zope-specific metadata, CVS stores
  the non-Zope specific data.

  Disadvantages: Metadata is not versioned in CVS, and remains only in
  the ZODB.  This means that the Zope environment is not reproducible
  from the information in CVS alone.  

  If:

  a) it is an absolute requirement to version all metadata, as well
  as all data, and

  b) you are starting from scratch and don't already have a large
  number of files in CVS,

  perhaps it would be better to use a product that "pickles" Zope
  products and stores their complete contents in CVS (data _and_
  metadata).  See
  "ZCVSFolder":http://www.zope.org/Members/sspickle/ZCVSMixin . The
  major disadvantage with ZCVSFolder, and the reason we did not use
  it, is that everything stored in CVS as "pickles," and is therefore
  not directly useable by other tools.

External Dependencies

  CVSFile calls out to CVS via standard Python system calls, and
  therefore requires the "cvs" commandline program to be installed and
  available in the PATH -- that is, the PATH used when starting up
  Zope (e.g. something like "python z2.py").  Fortunately, CVS is
  available for nearly all platforms from
  "cvshome":http://www.cvshome.org .  CVSFile doesn't require any
  particular version of CVS, and you should be able to use it on UNIX
  and windoze using either the regular or client/server flavor.

  **NOTE** 

  CVSFile does not currently provide any support for doing "cvs login"
  through the web.  Therefore, if you are using the client/server
  flavor of CVS, you will have to do a "cvs login" on the Zope server
  machine before executing any CVSFile commands.

CVS Sandbox Registry

  In order to enable parallel development as much as possible, we want
  to avoid strict locking such as required by various CVS+Web products
  like CVSwebClient.  They require strict locking in order to deal
  with possible race conditions as they provide direct access to the
  CVS repository itself.  Instead, we would like to point at a CVS
  sandbox.  What does this mean?  In order to avoid these locking
  issues, we would like each developer to access content from his own
  CVS sandbox, while using the central Zope server as per usual for
  navigation and non-CVS Zope objects.

  We have therefore created an object called a CVSSandboxRegistry.
  This object represents a logically related group of CVS sandboxes.
  Examples of logically related groups might be: the "development"
  group, or the "QA" group, but any grouping is fine.  Information
  about each of the sandboxes within the group is held in the
  registry object.  Every registry object must have one and only one
  sandbox designated as the "default sandbox."  A registry object
  contains information about at least one, and at most arbitrarily
  many sandboxes.

  In order to create an instance of CVSFile, or to perform CVS
  commands on an existing instance of CVSFile, or to view the contents
  of a CVSFile (either in the second view (ZMI) or third view (user
  web site)), there must exist an instance of CVSSandboxRegistry that
  is available via acquisition.  The CVSFile instance obtains
  the following information from the registry:

  - which sandbox to use (the "current" sandbox), and

  - the "current" sandbox base directory.

  How does a CVSFile instance find a CVSSandboxRegistry?

  The current lookup policy is as follows: use the first registry
  found by acquisition.  Alternatively, a more complex policy could be
  used such as 

  - looking in some predefined place, where the place could be
  specified in some property (a la Zope3 explicit acquisition), or

  - finding all registries in the ZODB and presenting a list for
  selection by the user.  

  Feel free to adopt either policy by reimplementing the function
  findCVSSandboxRegistry() in the CVSFile.py module.  YMMV

  **NOTE** Because of the way it is currently implemented, you
    _cannot_ create a CVSSandboxRegistry in the Zope Root Folder.
    Instead, it must be in a subfolder of ZODB.

Which Sandbox?

  The registry object enables a user to select a particular sandbox in
  which to work.  All CVSFile objects within the scope of this
  registry object will then operate with respect to the selected
  sandbox.  The information that identifies which sandbox is selected
  is held in a browser cookie.  If the cookie is not present for
  whatever reason, the "default" sandbox is automatically used.  This
  is the reason that there must always be exactly one sandbox within a
  CVSSandboxRegistry that is labelled the "default."

  The default sandbox is very important.  It is the one that content
  will be served out of for anyone who has _not_ made a sandbox
  selection, either because they decided not to, or because they do
  not have access to the ZMI.  For example, the general public!

  To create a registry object, the initial default CVS sandbox must be
  registered.  However, new CVS sandboxes may be added at any time,
  and the selection of which is the default can be changed at any
  time.

  The registry object allows the user to select a CVS sandbox in which
  to work, to change that selection at any time, and to perform CVS
  operations across the user's currently selected sandbox as a whole
  or, (eventually, when we implement it!) across all sandboxes within
  the group.
  
  When the user selects a sandbox in which to work, a browser cookie
  is automatically created that records this choice.  The cookie is
  given a very long "lifetime" by default, because developers or
  content authors usually want to work in the same sandbox every day.
  In this way, a developer may only have to select a sandbox once per
  project, or even less frequently.

  Confusion may result, however, if a developer changes to a different
  browser.  For example, say a developer normally uses Internet
  Explorer, and has made his sandbox selection within the ZMI using
  IE.  This means that IE is holding a cookie that records this
  selection.  If the developer fires up Opera and looks at the
  content, **without visiting the registry object to select his
  sandbox**, he will potentially see different content in the two
  browsers.  This is because the Zope "view" in IE is serving up
  CVSFile content from his selected sandbox, while the Zope "view" in
  Opera is serving up CVSFile content from the "default" sandbox.

  We are currently looking into ways to solve this problem.  Please
  feel free to forward any helpful suggestions.  The main issue is
  that we need a better understanding of typical usage patterns for
  developers and content authors.

How are files served up from sandboxes?

  CVSFile is a subclass of ExternalFile, so every instance of CVSFile
  is, of course, an ExternalFile.  As such, it records the pathname of the
  file to which it is a reference in two parts: basedir and filepath.

  CVSFile exploits this fact by equating sandbox location with
  basedir.  Each CVSFile resolves its file reference relative to the
  base directory of the selected sandbox.  The name of the selected
  sandbox is stored in a browser cookie, and the base directory of
  that sandbox is stored in a CVSSandboxRegistry.

  In this way, we could have a CVSFile that refered to:

  $basedir/com/arielpartners/knowledgebase/journal/j-20011017.xml

  Which might resolve to:

  /home/cstrong/ade/com/arielpartners/knowledgebase/journal/j-20011017.xml

  for me and:

  /home/zope/website-cvs/com/arielpartners/knowlegebase/journal/j-20011017.xml

  for a QA tester.  

  [ assuming $basedir means the sandbox base directory ]

  In this way, developers can use their local CVS sandboxes with
  their local Zope servers, or CVS sandboxes on a central Zope server,
  or both-- simply by pointing at different Zope servers and
  specifying different values for the sandbox name in their cookies.

  Local versions of the Zope database (ZODB) can be cloned from the
  server using the ZSyncer automated synchronization product.  See:

  "ZSyncer":http://www.zope.org/Members/andym/ZSyncer

  for more details.

Development Workflow

  CVSFile is intended for use in the development and testing phases of
  a project, for example to produce a corporate portal or website.
  This is when versioning and collaboration are especially important.
  Once a website has passed QA tests and is deemed ready for
  production, it should go through a release process.  That release
  process usually involves "freezing" or "snapshotting" the content.
  The versions of content files used are given "official versions"
  (e.g. cvs tags) so that they may be retrieved later.  In this way,
  the release could be recreated from scratch at a later time.

  In the future, CVSFile could support the release workflow process.
  For example: do a CVS tag of a set of versioned files, archive them,
  and perhaps import them into the ZODB for production (so there are
  no production dependencies on external filesystems-- beyond that of
  the Zope server itself).  The production versions would be "frozen,"
  so that edit functionality would be disabled.

Creating Instances Programmatically

  Normally instances of CVSFile are created by hand in the Zope
  Management Interface.  There are a number of screens (a
  mini-workflow) that are traversed in order to ensure that the
  instance is created properly, to deal with all of the things that
  can go wrong (creating new files, overwriting existing files,
  permission issues, etc.) and to confirm exactly what was done.

  However, there are times when you don't want that.  You simply want
  to run a Python script and create an instance of CVSFile.  Do not
  despair!  This is easy to do.  Check out the 'extras' subdirectory.
  You will find the source code for an example: 'uploadCreate'.  It is
  commented and should be easy to adapt to your uses.  Basically it
  calls the 'addCVSFile()' method in CVSFile.py' directly,
  bypassing the GUI creation workflow code.

Contributions

  We hope others find this code useful.  If you have extended or
  improved this product, please feel free to submit your changes to 
  "us":mailto:support@arielpartners.com.
  
  If there is enough interest, we would certainly consider setting up
  an open-source project on "SourceForge":http://www.sourceforge.net.

Schema Migration

  From time to time, you may find yourself with a new version of this
  product, either because we have released a "new improved" version or
  from some changes you may have made on your own.  How do you deal
  with all of the existing instances in your ZODB that were created
  with the old definition?  Here is our preferred technique:

  1. Find the repair() method (we usually put it at the bottom of the
     respective python source file)

  2. Change the repair() method so that it updates the object from the
     old version to the new version

  3. Create a python script in the root folder that recursively calls
     repair() on each of the object with a given metatype

  4. Execute the python script (the easy way is via the "test" tab)

  Here is our python script.  We call it "repairAll" and put it in the
  ZODB root folder::

    objects = context.ZopeFind(context, obj_metatypes=[metaType], search_sub=1)
    map(lambda x: x[1].repair(), objects)
    return map(lambda x: x[0], objects)

Notes

  This document is written in structured text.  For a quickie intro to
  structured text, look
  "here":http://www.zope.org/Documentation/Articles/STX.

Unit Testing on Win32

  If you don't plan to run the automated unit test suite, this
  section is irrelevant.

  For some reason, the automated unit tests don't run properly for me
  on win32 using the FAT file system under Zopes earlier than 2.5.1.
  I found the following workaround.  In the file name::

    [ZOPE]\lib\python\Testing\custom_zodb.py

  Where ZOPE stands for the directory in which you installed Zope,
  Change line number seven::

    Storage = DemoStorage(base=FileStorage(dfi,read_only=1), quota=(1<<20))

  to instead read::

    Storage = DemoStorage(base=FileStorage(dfi, read_only=0), quota=(1<<20))

  My guess is that FAT does not support the read_only attribute as
  required.
