sogo/SOPE/GDLContentStore
Wolfgang Sourdeau 72a7d598d5 Monotone-Parent: 3556596e54bd1f81160e5c10120b46259ec240f8
Monotone-Revision: fd483b959e9ace05c008817b99952497b2b47141

Monotone-Author: wsourdeau@inverse.ca
Monotone-Date: 2009-06-03T14:30:49
Monotone-Branch: ca.inverse.sogo
2009-06-03 14:30:49 +00:00
..
COPYING Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
COPYRIGHT Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
ChangeLog Monotone-Parent: 3556596e54bd1f81160e5c10120b46259ec240f8 2009-06-03 14:30:49 +00:00
EOAdaptorChannel+GCS.h Monotone-Parent: 5092b7192c90ae7b78ecc0442ac5ec1841d30a8d 2008-06-30 14:08:10 +00:00
EOAdaptorChannel+GCS.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
EOQualifier+GCS.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
EOQualifier+GCS.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
GCSChannelManager.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSChannelManager.m Monotone-Parent: f845aee937de02ccc5aaec9e4f9592ddeea27abf 2009-03-20 20:35:39 +00:00
GCSContext.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSContext.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
GCSFieldExtractor.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSFieldExtractor.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
GCSFieldInfo.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSFieldInfo.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
GCSFolder.h Monotone-Parent: fd50c862e356e0c0c102bf3b5901ec2e9de8241e 2009-06-02 16:22:28 +00:00
GCSFolder.m Monotone-Parent: fd50c862e356e0c0c102bf3b5901ec2e9de8241e 2009-06-02 16:22:28 +00:00
GCSFolderManager.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSFolderManager.m Monotone-Parent: 3556596e54bd1f81160e5c10120b46259ec240f8 2009-06-03 14:30:49 +00:00
GCSFolderType.h Monotone-Parent: a3fc297161b789eefee18087844b47a8aaf75178 2009-03-24 16:33:08 +00:00
GCSFolderType.m Monotone-Parent: a3fc297161b789eefee18087844b47a8aaf75178 2009-03-24 16:33:08 +00:00
GCSStringFormatter.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
GCSStringFormatter.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
GNUmakefile See ChangeLog 2009-04-01 13:49:24 +00:00
GNUmakefile.preamble Monotone-Parent: d8ee782e0279aa96226ceac45acca9d744f3f234 2008-02-14 17:31:23 +00:00
NSURL+GCS.h Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
NSURL+GCS.m Monotone-Parent: 7693a13871fea2b066790d804159529866a7e4c8 2008-12-12 17:57:23 +00:00
README Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00
Version Adjusted the version number 2009-05-21 16:44:40 +00:00
fhs.make Monotone-Parent: 70da0038473c195e4432eef83ea8e60444efb5e6 2007-10-29 14:51:55 +00:00

README

Storage Backend
===============

The storage backend implements the "low level" folder abstraction, which is
basically an arbitary "BLOB" containing some document. The feature is that
we extract "quick access" / "searchable" attributes from the document content.

Further it contains the "folder management" API, as named folders can be stored
in different databases.
Note: we need a way to tell where "new" folders should be created
Note: to sync with LDAP we need to periodically delete or archive old folders

Folders have associated a type (like 'calendar') which defines the query
attributes and serialization format.

TODO
====
- fix some OCS naming
  - defaults
  - lookup directories
- hierarchies deeper than 4 (properly filter on path in OCS)

Open Questions
==============

System-meta-data in the blob-table or in the quick-table?
- master data belongs into the blob table
- could be regular 'NSxxx' keys to differentiate meta data from

Class Hierarchy
===============

  [NSObject]
    OCSContext                  - tracking context
    OCSFolder                   - represents a single folder
    OCSFolderManager            - manages folders
    OCSFolderType               - the mapping info for a specific folder-type
    OCSFieldInfo                - mapping info for one 'quick field'
    OCSChannelManager           - maintains EOAdaptorChannel objects

  TBD:
  - field 'extractor'
  - field 'value' (eg array values for participants?)
  - BLOB archiver/unarchiver

Defaults
========

  OCSFolderInfoURL - the DB URL where the folder-info table is located
    eg: http://OGo:OGo@localhost/test/folder_info

  OCSFolderManagerDebugEnabled      - enable folder-manager debug logs
  OCSFolderManagerSQLDebugEnabled   - enable folder-manager SQL gen debug logs

  OCSChannelManagerDebugEnabled     - enable channel debug pooling logs
  OCSChannelManagerPoolDebugEnabled - debug pool handle allocation
  
  OCSChannelExpireAge       - if that age in seconds is exceeded, a channel 
                              will be removed from the pool
  OCSChannelCollectionTimer - time in seconds. each n-seconds the pool will be
			      checked for channels too old
  
  [PGDebugEnabled] - enable PostgreSQL adaptor debugging

URLs
====

  "Database URLs"
  
  We use the schema:
    postgresql://[user]:[password]@[host]:[port]/[dbname]/[tablename]

Support Tools
=============

- tools we need:
  - one to recreate a quick table based on the blob table

Notes
=====

- need to use http:// URLs for connect info, until generic URLs in 
  libFoundation are fixed (the parses breaks on the login/password parts)

QA
==

Q: Why do we use two tables, we could store the quick columns in the blob?
==
They could be in the same table. We considered using separate tables since the 
quick table is likely to be recreated now and then if BLOB indexing 
requirements change.
Actually one could even use different _quick tables which share a common BLOB 
table.
(a quick table is nothing more than a database index and like with DB indexes 
 multiple ones for different requirements can make sense).

Further it might improve caching behaviour for row based caches (the quick 
table is going to be queried much more often) - not sure whether this is 
relevant with PostgreSQL, probably not?

Q: Can we use a VARCHAR primary key?
==
We asked in the postgres IRC channel and apparently the performance penalty of
string primary keys isn't big.
We could also use an 'internal' int sequence in addition (might be useful for
supporting ZideLook)
Motivation: the 'iCalendar' ID is a string and usually looks like a GUID.

Q: Why using VARCHAR instead of TEXT in the BLOB?
==
To quote PostgreSQL documentation:
"There are no performance differences between these three types, apart from 
 the increased storage size when using the blank-padded type."
So varchar(xx) is just a large TEXT. Since we intend to store mostly small
snippets of data (tiny XML fragments), we considered VARCHAR the more 
appropriate type.