External Database Drivers
Chapter 12-7
External Database Drivers
This chapter describes how to write a Domino database driver. A database driver allows a Domino or Notes user to read data from a non-Domino database by calling the functions @DbLookup and @DbColumn. Before reading this chapter, you should have a good understanding of the user-level functionality and syntax of the @DbLookup and @DbColumn functions, as described in the Domino 7.0 Designer Help documentation.
The code in this chapter is intended as a general illustration of the ideas discussed. For a more detailed look at a simple database driver, see the sample program samples\misc\dbdrive.
Background
The Domino Designer functions, @DbLookup and @DbColumn, are basically context-free, to keep their syntax simple. That is, the user is never asked to "Open" or "Close" a database in formulas that use these functions; furthermore, they are executed in such rapid succession that caching must be implemented at several levels to gain any efficiency.
The highest layer of caching, Query/Result caching, is performed above the level of the database driver. For the most part, the semantics of the @DbLookup and @DbColumn functions mean that the higher-level caching is transparent to the driver, so it is not discussed in this chapter.
The other layer of caching, open database session management, is performed by the driver in cooperation with the higher layer of software, the Database Driver Manager. This manager is responsible for taking database @Function requests from the Formula Evaluator, parsing out the database driver name (in the first argument), loading the driver into the process's address space and initializing it if not already loaded, opening and managing sessions to the driver, and finally issuing the Function requests to the driver itself.
The driver itself has two principal responsibilities: managing "sessions" with the outside world and performing Functions on a session as requested. The following explains the various driver functions in more detail and provides some skeleton subroutines to use as a guide.
Database Driver Packaging and Concurrency Requirements
As in all Domino and Notes low-level subsystems, database drivers must obey strict rules to fit into their computing environment:
- The drivers are packaged as executable program libraries (Dynamic Link Libraries under Windows, shared objects under UNIX).
2. You must decide on a mnemonic class name for the database format that your driver will access. The class name can be up to five characters long. Preferably, it is nationality-neutral (obscure in all languages); for example, DBASE. You supply the class name as the first parameter when you call @DbLookup and @DbColumn.
3. The file name for a database driver must be in the following format: a platform-dependent prefix, followed by the class name, followed by a platform-dependent file extension (if required). The prefixes and extensions required for each platform are as follows:
Windows 32-bit: prefix: "ndb" file extension: ".dll"
AIX: prefix: "libdb" file extension: ".a"
Linux: prefix "libdb" file extension: ".so"
Therefore, for a database with the class name "abc", the filenames for the database driver on the various platforms would be:
Windows 32-bit: "ndbabc.dll"
AIX: "libdbabc.a"
Linux: "libdbabc.so"
To install the database driver, place its executable program library into the Domino or Notes program directory.
4. Under Windows, the module definition file (.DEF file) must specify the data as being MOVEABLE SINGLE so that it doesn't occupy GlobalDOS memory and because Windows doesn't support multi-instance dynamic link libraries.
5. Because of instantiation requirements of the operating systems, the library is loaded once per process. At load time under Windows, the first entry point address (entry point @1) is obtained by ordinal and the driver's Init function is invoked. Under UNIX, use MainEntryPoint for the driver's Init function. The principal use of this function is to plug a data structure with vectors to other database driver subroutines, including its Term function, which will be called by Domino or Notes when the process exits or calls NotesTerm.
- Remember that in some environments (such as Windows), you must export in your DEF file all of your functions that Domino or Notes will call through these vectors. This is because these environments use the fact that it is exported to generate a thunk that sets up the program's DS appropriately on all of these entry points. Under AIX, you must create an export file and define the exported symbols.
6. As in all Domino and Notes subsystems, all entry points to the drivers must be completely reentrant, both by multiple threads in a single process and by multiple processes. In Windows, reentrancy is also a requirement, since if you call Yield, you may also be preempted by another process that does a database driver call. (Domino and Note makes heavy use of multiple threads and multiple processes, so treat this as a requirement, not as something that can be done "later".)
7. As you will note later, the database driver is responsible for maintaining queues (or arrays) of data structures internally. Therefore, be aware that shared memory in UNIX does not exist at the same place in address spaces of multiple processes, so you cannot have address-based queues in shared memory. If you use MULTIPLE on your executable program libraries and keep such queues in the local heap, or use non-shared memory for your queues, you should be safe.
Database Driver Header Files and Error Status Codes
The database driver needs to include two C API header files. dbdrv.h contains the definition of the database driver vector structure filled in by the initialization routine and miscellaneous constants. dbdrverr.h contains error status code definitions that span all drivers. If a database driver must return error codes not contained in dbdrverr.h, the codes should be in the range 0x31B4 (PKG_DBD+180) through 0x31B7 (PKG_DBD+183) inclusive, so that they don't overlap other Domino and Notes error codes.
Database Driver Initialization and Termination
The driver's library is loaded once per process. At load time under Windows, the first entry point address (entry point @1) is obtained by ordinal and the driver's Init function is invoked. Under UNIX, the entry point, which is the driver's Init function, must be called MainEntryPoint.
The principal use of the driver's Init function is to return a vector of other driver subroutine addresses, defined in the DBVEC data structure in the header file dbdrv.h. Your driver should not define a function for the SetOpenContext member of the DBVEC data structure. This function is reserved for use by Domino and Notes.
Because the database driver's Init function is passed a context block containing the driver's module handle, you can also use it to load strings or other resources from the driver's module.
NotesTerm invokes the driver's Term function just prior to this process's exit. The higher layers of software guarantee that it will already have asked the driver to close all open databases/sessions prior to calling Term, so the Term routine probably needs to do nothing.
You can use the following code as a template for the Init and Term functions for a database driver.
NOTE: The format of the session queue is defined by the database driver, not by Domino or Notes. This code shows one way to initialize a session queue.
|
/ Initialize the database driver
* Do per-process initialization. This is called just after
* the LoadLibrary and is the first entry point in the library.
* When this entry point is called, all the other entry point
* vectors are filled in by this routine.
* Inputs:
* drv - database driver vector
* Outputs:
* (routine) - error status
* drv vectors to other subroutines have been filled in
/
STATUS LNPUBLIC MainEntryPoint(DBVEC drv)
{
/ Fill in the subroutine vectors /
drv->Init = MainEntryPoint;
drv->Term = DBDTerm;
drv->Open = DBDOpen;
drv->Close = DBDClose;
drv->Function = DBDPerformFunction;
/ Perform one-time initializations by initializing local
session queue if it hasn't yet been initialized. (In
Windows, with single-instance DS, this is a queue shared
by all processes. In all other environments, it's
process-private.
/
if (SessionQueue.Next == NULL)
SessionQueue.Next = SessionQueue.Prev = &SessionQueue;
/ Done /
return(NOERROR);
}
/ Terminate the database driver
* Per-process termination routine, ASSUMING that all open
* sessions for this context have been closed by the time
* this is called. This is called just prior to the
FreeLibrary.
* Inputs:
* drv - database driver vector
* Outputs:
* (routine) - error status
/
STATUS LNPUBLIC DBDTerm(DBVEC drv)
{
/ Your per-process termination code here */
return(NOERROR);
}
Database Driver Sessions
Because of the semantics of the @Db functions, Domino and Notes request the database driver to perform @DbLookup and @DbColumn functions without any "handle" to an open database table. Furthermore, it expects that the operation will be extremely fast, so it doesn't want the driver to open and close the database on every Function request. For this reason Domino and Notes, in cooperation with the drivers, implements the abstraction of a "session."
Before issuing a function request from a thread, Domino or Notes ask the database driver to open a session. At this point, the driver merely allocates a small data structure and returns an opaque handle to its caller, which Domino or Notes will then pass to the driver on future function requests. Every time the driver has to open a database in response to a function request, the open database context is maintained by enqueueing it off the session data structure. Future function requests on that session use the same, already-open database context.
It is important to understand the usage and scope of a session. First, Domino or Notes guarantees that a session, once opened, will only be used by the thread that requested the open. That is, Domino and Notes do not pass session handles between processes or threads. Second, a single thread may have multiple sessions open at any given time. Third, functions will be performed on the same database in multiple sessions, perhaps by the same thread and perhaps by a different thread. Last, sessions stay open for a long time. They open when a user double-clicks on a database icon and close when a user closes the database, which could be much later.
NOTE: Since the maintenance of session queues and database contexts is the responsibility of the database driver, their design is up to the designer of the driver. For some applications, dynamically-allocated linked lists might be the best solution; for others, a statically-defined array might be more appropriate.
|
/ Open a session
* Any databases opened as a side effect of Functions
* performed on this session will gather up their context
* in the hSession returned by this routine.
* Inputs:
* drv - database driver vector
* Outputs:
* rethSession = filled in with session handle
* (routine) - error status
/
STATUS LNPUBLIC DBDOpen(DBVEC drv, HDBDSESSION rethSession)
{
STATUS error;
SESSION sess;
/ Allocate a new, empty session context block and enqueue it. /
if (error = Alloc(sizeof(SESSION), &sess))
return(error);
sess->DatabaseQueue.Next = sess->DatabaseQueue.Prev =
&sess->DatabaseQueueQueue;
Enqueue(&SessionQueue, sess);
/ Return it to the caller as an opaque handle. /
rethSession = (HDBDSESSION) sess;
return(NOERROR);
}
/ Close an open session
* Close a session, and as a side effect all databases whose context
* has been built up in hSession.
* Inputs:
* drv - database driver vector
* hSession - session handle to close
* Outputs:
* (routine) - error status
/
STATUS LNPUBLIC DBDClose(DBVEC drv, HDBDSESSION hSession)
{
SESSION sess = (SESSION ) hSession;
/ Close and deallocate all open database/table descriptors /
while (!QueueIsEmpty(&sess->DatabaseQueue))
{
DATABASE db = (DATABASE ) &sess->DatabaseQueue.Next;
CloseDatabaseTable(sess, db);
Dequeue(db);
Free(db);
}
/ Dequeue and deallocate the session itself. /
Dequeue(sess);
Free(sess);
/ Done /
return(NOERROR);
}
Database Driver Functions
A single entry point to the driver performs all database functions. Currently, the only supported database functions are @DbLookup and @DbColumn. The function entry point is a generalized argc/argv interface, except that the arguments are Domino data types, not C char *'s, and there is an "argl" vector containing the lengths of the objects at which the argv's point. If the function executes without error, it must allocate a Domino memory object (OSMemAlloc) containing the resulting Domino data. Correlation between the input argument data types and the output data type is not necessary.
Remember that the first WORD of Domino data is a standard Domino data type, followed by the actual data. (For more information, see the Reference.) Anticipate and handle the following data types by converting them as necessary to the type you need for your query. If you receive any other data type, you can fail the query and return the error ERR_DBD_DATATYPE. As always in Domino and Notes, all text is in LMBCS.
TYPE_TEXT
TYPE_TEXT_LIST
TYPE_NUMBER
TYPE_NUMBER_RANGE
TYPE_TIME
TYPE_TIME_RANGE
The actual format of the content of the arguments is up to the database driver, although the sample code below follows the convention of using the first available argument (the database driver name having already been stripped off) as the database file name and the second argument as the table name. For instance, if the design of the database accessed by your driver does not include the notion of a table, you can us the second argument to specify some other information that applies to your database. Of course, you must document this change for users of your database driver.
|
/ Perform a database function
* Perform a function on a session. If any databases must be opened
* as a side effect of this function, gather context into hSession
* so that it may be later deallocated/closed in Close.
* Inputs:
* drv - database driver vector
* Function - function ID
* argc - number of arguments
* argl - array of argument lengths
* argv - array of argument pointers
* Outputs:
* rethResult = filled in with result handle
* retResultLength = filled in with result buffer length
* (routine) - error status
/
STATUS LNPUBLIC DBDPerformFunction(DBVEC drv, HDBDSESSION hSession,
WORD Function,
WORD argc, DWORD argl, void argv,
HANDLE rethResult, DWORD retResultLength)
{
SESSION sess = (SESSION ) hSession;
DATABASE db;
STATUS error;
char DbName;
WORD DbNameLength,
char TableName;
WORD TableNameLength,
/ Check args and extract database/table name /
if (argc < 2)
return(ERR_DBD_INSUFF_ARGS);
DbName = argv[0];
DbNameLength = (WORD) argl[0];
TableName = argv[1];
TableNameLength = (WORD) argl[1]);
argc -= 2;
argv += 2;
argl += 2;
/ Find an instance of the open database on this session. /
for (db = (DATABASE ) &sess->DatabaseQueue.Next;
!IsQueueHead(&sess->DatabaseQueue, db);
db = (DATABASE ) db->Links.Next))
{
if (DbNameLength != db->DbNameLength)
continue;
if (TableNameLength != db->TableNameLength)
continue;
if (memcmp(DbName, db->DbName, DbNameLength) != 0)
continue;
if (memcmp(TableName, db->TableName, TableNameLength) != 0)
continue;
break;
}
/ If not found on queue, open and enqueue it. /
if (IsQueueHead(&sess->DatabaseQueue, db))
{
if (error = Alloc(sizeof(DATABASE), &db))
return(error);
db->DbName = DbName;
db->DbNameLength = DbNameLength;
db->TableName = TableName;
db->TableNameLength = TableNameLength;
if (error = OpenDatabaseTable(sess, db))
{
Free(db);
return(error);
}
Enqueue(&sess->DatabaseQueue, db);
}
/ Dispatch based upon function /
switch (Function)
{
/ Process LOOKUP /
case DB_LOOKUP:
error = Lookup(db, argc, argl, argv,
rethResult, retResultLength);
break;
/ Process COLUMN /
case DB_COLUMN:
error = Column(db, argc, argl, argv,
rethResult, retResultLength);
break;
/ Unknown function /
default:
error = ERR_DBD_FUNCTION;
}
/ Done */
return(error);
}
Query/Result Caching
The higher layers of software in Domino and Notes outside of the database driver cache complete queries and their results for moderate periods. You can override this caching by using a special "nocache" qualifier, but this is usually unnecessary and unused. As long as the database driver's implementation of @DbLookup and @DBColumn has no side effects (and it shouldn't, given the semantics), caching should work well across driver implementations.