Gadfly (database)

Gadfly is a relational database management system written in Python. Gadfly is a collection of Python modules that provides relational database functionality entirely implemented in Python. It supports a subset of the standard RDBMS Structured Query Language (SQL). Gadfly runs wherever Python runs and supports client/server on any platform that supports the standard Python socket interface. The file formats used by Gadfly for storage are cross platform—a gadfly database directory can be moved from Windows 95 to Linux using a binary copying mechanism and gadfly will read and run the database. It supports persistent databases consisting of a collection of structured tables with indices, and a large subset of SQL for accessing and modifying those tables. It supports a log based recovery protocol which allows committed operations of a database to be recovered even if the database was not shut down in a proper manner (i. e., in the event of a CPU or software crash, [but not in the event of a disk crash]). It also supports a TCP/IP Client/Server mode where remote clients can access a Gadfly database over a TCP/IP network (such as the Internet) subject to configurable security mechanisms. Since Gadfly depends intimately on the kwParsing package it is distributed as part of the kwParsing package, under the same copyright. Gadfly allows Python programs to store, retrieve and query tabular data without having to rely on any external database engine or package. It provides an in memory relational database style engine for Python programs, complete with a notion of a "committed, recoverable transaction" and "aborts".

The main "gadfly" module attempts to faithfully adhere to Greg Stein's Python Database API, as discussed and certified by the Python DB SIG. Concurrent database updates are not supported. The "databases" are currently designed to be written/modified by one process in isolation. Multiple processes can access a Gadfly database when accesses are arbitrated by a TCP/IP Gadfly server process.

In this version all tables are read into memory upon "connecting" to the database and "touched" tables are written out upon checkpoint. Each table is represented as a separate file in the destination directory, and there is a "data definition" file as well (a list of data definition declarations). During active use a log file appears in the active directory as well, and if the process crashes this log file is used to recover committed operations.

The SQL grammar is described in grammar. py, the binding of the grammar constructs to semantic objects is performed in bindings. py, the semantic objects and their execution strategies is defined in semantics. py. The semantics use a lot of classical and non classical logic (cylindric logic) as well as optimization heuristics to define a relatively efficient and correct implementation of SQL. The most basic data structures of the implementation are given in either kjbuckets0. py or the faster kjbucketsmodule. c, which implement the same data type signatures in Python and in a C extension to Python respectively. The database. py module is a simple wrapper that provides a standard DBAPI interface to the system. The test suite test/test gadfly. py attempts to provide a regression test and a demonstration of the system. The SQL parser also requires the kwParsing parser generation package, which consists of a number of additional python modules.

With gfplus a gadfly database can be interactively manipulated with SQL commands. The tool works similar to Oracle's SQL*Plus.

Because it lacks true concurrency control and file system based indexing it is not appropriate for very large multiprocess transaction based systems. Two applications may access the same database concurrently. However, changes made by one application may not be seen by the other application until after it restarts. This may be because each application loads the database in memory at startup only.