.\" Man page generated from reStructuredText. . . .nr rst2man-indent-level 0 . .de1 rstReportMargin \\$1 \\n[an-margin] level \\n[rst2man-indent-level] level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] - \\n[rst2man-indent0] \\n[rst2man-indent1] \\n[rst2man-indent2] .. .de1 INDENT .\" .rstReportMargin pre: . RS \\$1 . nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] . nr rst2man-indent-level +1 .\" .rstReportMargin post: .. .de UNINDENT . RE .\" indent \\n[an-margin] .\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] .nr rst2man-indent-level -1 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] .in \\n[rst2man-indent\\n[rst2man-indent-level]]u .. .TH "SQLALCHEMY-CONTINUUM" "1" "Apr 08, 2024" "1.3.14" "SQLAlchemy-Continuum" .SH NAME sqlalchemy-continuum \- SQLAlchemy-Continuum Documentation .sp SQLAlchemy\-Continuum is a versioning extension for SQLAlchemy. .SH INTRODUCTION .SS Why? .sp SQLAlchemy \fI\%already has a versioning extension\fP\&. This extension however is very limited. It does not support versioning entire transactions. .sp Hibernate for Java has Envers, which had nice features but lacks a nice API. Ruby on Rails has \fI\%papertrail\fP, which has very nice API but lacks the efficiency and feature set of Envers. .sp As a Python/SQLAlchemy enthusiast I wanted to create a database versioning tool for Python with all the features of Envers and with as intuitive API as papertrail. Also I wanted to make it _fast_ keeping things as close to the database as possible. .SS Features .INDENT 0.0 .IP \(bu 2 Does not store updates which don\(aqt change anything .IP \(bu 2 Supports alembic migrations .IP \(bu 2 Can revert objects data as well as all object relations at given transaction even if the object was deleted .IP \(bu 2 Transactions can be queried afterwards using SQLAlchemy query syntax .IP \(bu 2 Querying for changed records at given transaction .IP \(bu 2 Querying for versions of entity that modified given property .IP \(bu 2 Querying for transactions, at which entities of a given class changed .IP \(bu 2 History models give access to parent objects relations at any given point in time .UNINDENT .SS Installation .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C pip install SQLAlchemy\-Continuum .ft P .fi .UNINDENT .UNINDENT .SS Basics .sp In order to make your models versioned you need two things: .INDENT 0.0 .IP 1. 3 Call make_versioned() before your models are defined. .IP 2. 3 Add __versioned__ to all models you wish to add versioning to .UNINDENT .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C import sqlalchemy as sa from sqlalchemy_continuum import make_versioned make_versioned(user_cls=None) class Article(Base): __versioned__ = {} __tablename__ = \(aqarticle\(aq id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) name = sa.Column(sa.Unicode(255)) content = sa.Column(sa.UnicodeText) # after you have defined all your models, call configure_mappers: sa.orm.configure_mappers() .ft P .fi .UNINDENT .UNINDENT .sp After this setup SQLAlchemy\-Continuum does the following things: .INDENT 0.0 .IP 1. 3 It creates ArticleHistory model that acts as version history for Article model .IP 2. 3 Creates TransactionLog and TransactionChanges models for transactional history tracking .IP 3. 3 Adds couple of listeners so that each Article object insert, update and delete gets recorded .UNINDENT .sp When the models have been configured either by calling configure_mappers() or by accessing some of them the first time, the following things become available: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import version_class, parent_class version_class(Article) # ArticleHistory class parent_class(version_class(Article)) # Article class .ft P .fi .UNINDENT .UNINDENT .SS Versions and transactions .sp At the end of each transaction SQLAlchemy\-Continuum gathers all changes together and creates version objects for each changed versioned entity. Continuum also creates one TransactionLog entity and N number of TransactionChanges entities per transaction (here N is the number of affected classes per transaction). TransactionLog and TransactionChanges entities are created for transaction tracking. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqSome article\(aq) session.add(article) session.commit() article.versions[0].name == u\(aqSome article\(aq article.name = u\(aqSome updated article\(aq session.commit() article.versions[1].name == u\(aqSome updated article\(aq .ft P .fi .UNINDENT .UNINDENT .SH VERSION OBJECTS .SS Operation types .sp When changing entities and committing results into database Continuum saves the used operations (INSERT, UPDATE or DELETE) into version entities. The operation types are stored by default to a small integer field named \(aqoperation_type\(aq. Class called \(aqOperation\(aq holds convenient constants for these values as shown below: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import Operation article = Article(name=u\(aqSome article\(aq) session.add(article) session.commit() article.versions[0].operation_type == Operation.INSERT article.name = u\(aqSome updated article\(aq session.commit() article.versions[1].operation_type == Operation.UPDATE session.delete(article) session.commit() article.versions[2].operation_type == Operation.DELETE .ft P .fi .UNINDENT .UNINDENT .SS Version traversal .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C first_version = article.versions[0] first_version.index # 0 second_version = first_version.next assert second_version == article.versions[1] second_version.previous == first_version # True second_version.index # 1 .ft P .fi .UNINDENT .UNINDENT .SS Changeset .sp Continuum provides easy way for getting the changeset of given version object. Each version contains a changeset property which holds a dict of changed fields in that version. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqNew article\(aq, content=u\(aqSome content\(aq) session.add(article) session.commit(article) version = article.versions[0] version.changeset # { # \(aqid\(aq: [None, 1], # \(aqname\(aq: [None, u\(aqNew article\(aq], # \(aqcontent\(aq: [None, u\(aqSome content\(aq] # } article.name = u\(aqUpdated article\(aq session.commit() version = article.versions[1] version.changeset # { # \(aqname\(aq: [u\(aqNew article\(aq, u\(aqUpdated article\(aq], # } session.delete(article) version = article.versions[1] version.changeset # { # \(aqid\(aq: [1, None] # \(aqname\(aq: [u\(aqUpdated article\(aq, None], # \(aqcontent\(aq: [u\(aqSome content\(aq, None] # } .ft P .fi .UNINDENT .UNINDENT .sp SQLAlchemy\-Continuum also provides a utility function called changeset. With this function you can easily check the changeset of given object in current transaction. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import changeset article = Article(name=u\(aqSome article\(aq) changeset(article) # {\(aqname\(aq: [None, u\(aqSome article\(aq]} .ft P .fi .UNINDENT .UNINDENT .SS Version relationships .sp Each version object reflects all parent object relationships. You can think version object relations as \(aqrelations of parent object in given point in time\(aq. .sp Lets say you have two models: Article and Category. Each Article has one Category. In the following example we first add article and category objects into database. .sp Continuum saves new ArticleVersion and CategoryVersion records in the background. After that we update the created article entity to use another category. Continuum creates new version objects accordingly. .sp Lastly we check the category relations of different article versions. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C category = Category(name=u\(aqSome category\(aq) article = Article( name=u\(aqSome article\(aq, category=category ) session.add(article) session.commit() article.category = Category(name=u\(aqSome other category\(aq) session.commit() article.versions[0].category.name # u\(aqSome category\(aq article.versions[1].category.name # u\(aqSome other category\(aq .ft P .fi .UNINDENT .UNINDENT .sp The logic how SQLAlchemy\-Continuum builds these relationships is within the RelationshipBuilder class. .SS Relationships to non\-versioned classes .sp Let\(aqs take previous example of Articles and Categories. Now consider that only Article model is versioned: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class Article(Base): __tablename__ = \(aqarticle\(aq __versioned__ = {} id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255), nullable=False) class Category(Base): __tablename__ = \(aqtag\(aq id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255)) article_id = sa.Column(sa.Integer, sa.ForeignKey(Article.id)) article = sa.orm.relationship( Article, backref=sa.orm.backref(\(aqcategories\(aq) ) .ft P .fi .UNINDENT .UNINDENT .sp Here Article versions will still reflect the relationships of Article model but they will simply return Category objects instead of CategoryVersion objects: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C category = Category(name=u\(aqSome category\(aq) article = Article( name=u\(aqSome article\(aq, category=category ) session.add(article) session.commit() article.category = Category(name=u\(aqSome other category\(aq) session.commit() version = article.versions[0] version.category.name # u\(aqSome other category\(aq isinstance(version.category, Category) # True .ft P .fi .UNINDENT .UNINDENT .SS Dynamic relationships .sp If the parent class has a dynamic relationship it will be reflected as a property which returns a query in the associated version class. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class Article(Base): __tablename__ = \(aqarticle\(aq __versioned__ = {} id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255), nullable=False) class Tag(Base): __tablename__ = \(aqtag\(aq __versioned__ = {} id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255)) article_id = sa.Column(sa.Integer, sa.ForeignKey(Article.id)) article = sa.orm.relationship( Article, backref=sa.orm.backref( \(aqtags\(aq, lazy=\(aqdynamic\(aq ) ) article = Article() article.name = u\(aqSome article\(aq article.content = u\(aqSome content\(aq session.add(article) session.commit() tag_query = article.versions[0].tags tag_query.all() # return all tags for given version tag_query.count() # return the tag count for given version .ft P .fi .UNINDENT .UNINDENT .SH REVERTING CHANGES .sp One of the major benefits of SQLAlchemy\-Continuum is its ability to revert changes. .SS Revert update .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqNew article\(aq, content=u\(aqSome content\(aq) session.add(article) session.commit(article) version = article.versions[0] article.name = u\(aqUpdated article\(aq session.commit() version.revert() session.commit() article.name # u\(aqNew article\(aq .ft P .fi .UNINDENT .UNINDENT .SS Revert delete .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqNew article\(aq, content=u\(aqSome content\(aq) session.add(article) session.commit(article) version = article.versions[0] session.delete(article) session.commit() version.revert() session.commit() # article lives again! session.query(Article).first() .ft P .fi .UNINDENT .UNINDENT .SS Revert relationships .sp Sometimes you may have cases where you want to revert an object as well as some of its relation to certain state. Consider the following model definition: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class Article(Base): __tablename__ = \(aqarticle\(aq __versioned__ = {} id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255)) class Tag(Base): __tablename__ = \(aqtag\(aq __versioned__ = {} id = sa.Column(sa.Integer, autoincrement=True, primary_key=True) name = sa.Column(sa.Unicode(255)) article_id = sa.Column(sa.Integer, sa.ForeignKey(Article.id)) article = sa.orm.relationship(Article, backref=\(aqtags\(aq) .ft P .fi .UNINDENT .UNINDENT .sp Now lets say some user first adds an article with couple of tags: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article( name=u\(aqSome article\(aq, tags=[Tag(u\(aqGood\(aq), Tag(u\(aqInteresting\(aq)] ) session.add(article) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp Then lets say another user deletes one of the tags: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C tag = session.query(Tag).filter_by(name=u\(aqInteresting\(aq) session.delete(tag) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp Now the first user wants to set the article back to its original state. It can be achieved as follows (notice how we use the relations parameter): .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = session.query(Article).get(1) article.versions[0].revert(relations=[\(aqtags\(aq]) session.commit() .ft P .fi .UNINDENT .UNINDENT .SH QUERIES .sp You can query history models just like any other sqlalchemy declarative model. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import version_class ArticleVersion = version_class(Article) session.query(ArticleVersion).filter_by(name=u\(aqsome name\(aq).all() .ft P .fi .UNINDENT .UNINDENT .SS How many transactions have been executed? .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import transaction_class Transaction = transaction_class(Article) Transaction.query.count() .ft P .fi .UNINDENT .UNINDENT .SS Querying for entities of a class at a given revision .sp In the following example we find all articles which were affected by transaction 33. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C session.query(ArticleVersion).filter_by(transaction_id=33) .ft P .fi .UNINDENT .UNINDENT .SS Querying for transactions, at which entities of a given class changed .sp In this example we find all transactions which affected any instance of \(aqArticle\(aq model. This query needs the TransactionChangesPlugin. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C TransactionChanges = Article.__versioned__[\(aqtransaction_changes\(aq] entries = ( session.query(Transaction) .innerjoin(Transaction.changes) .filter( TransactionChanges.entity_name.in_([\(aqArticle\(aq]) ) ) .ft P .fi .UNINDENT .UNINDENT .SS Querying for versions of entity that modified given property .sp In the following example we want to find all versions of Article class which changed the attribute \(aqname\(aq. This example assumes you are using PropertyModTrackerPlugin. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C ArticleVersion = version_class(Article) session.query(ArticleHistory).filter(ArticleVersion.name_mod).all() .ft P .fi .UNINDENT .UNINDENT .SH TRANSACTIONS .SS Transaction .sp For each committed transaction SQLAlchemy\-Continuum creates a new Transaction record. .sp Transaction can be queried just like any other sqlalchemy declarative model. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import transaction_class Transaction = transaction_class(Article) # find all transactions session.query(Transaction).all() .ft P .fi .UNINDENT .UNINDENT .SS UnitOfWork .sp For each database connection SQLAlchemy\-Continuum creates an internal UnitOfWork object. Normally these objects are created at before flush phase of session workflow. However you can also force create unit of work before this phase. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C uow = versioning_manager.unit_of_work(session) .ft P .fi .UNINDENT .UNINDENT .sp Transaction objects are normally created automatically at before flush phase. If you need access to transaction object before the flush phase begins you can do so by calling the create_transaction method of the UnitOfWork class. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C transaction = uow.create_transaction(session) .ft P .fi .UNINDENT .UNINDENT .sp The version objects are normally created during the after flush phase but you can also force create those at any time by calling make_versions method. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C uow.make_versions(session) .ft P .fi .UNINDENT .UNINDENT .SS Workflow internals .sp Consider the following code snippet where we create a new article. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article() article.name = u\(aqSome article\(aq article.content = u\(aqSome content\(aq session.add(article) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp This would execute the following SQL queries (on PostgreSQL) .INDENT 0.0 .IP 1. 3 .INDENT 3.0 .TP .B INSERT INTO article (name, content) VALUES (?, ?) params: (\(aqSome article\(aq, \(aqSome content\(aq) .UNINDENT .IP 2. 3 .INDENT 3.0 .TP .B INSERT INTO transaction (issued_at) VALUES (?) params: (datetime.utcnow()) .UNINDENT .IP 3. 3 .INDENT 3.0 .TP .B INSERT INTO article_version (id, name, content, transaction_id) VALUES (?, ?, ?, ?) params: (
, \(aqSome article\(aq, \(aqSome content\(aq, ) .UNINDENT .UNINDENT .SH NATIVE VERSIONING .sp As of version 1.1 SQLAlchemy\-Continuum supports native versioning for PostgreSQL dialect. Native versioning creates SQL triggers for all versioned models. These triggers keep track of changes made to versioned models. Compared to object based versioning, native versioning has .INDENT 0.0 .IP \(bu 2 Much faster than regular object based versioning .IP \(bu 2 Minimal memory footprint when used alongside \fIcreate_tables=False\fP and \fIcreate_models=False\fP configuration options. .IP \(bu 2 More cumbersome database migrations, since triggers need to be updated also. .UNINDENT .SS Usage .sp For enabling native versioning you need to set \fInative_versioning\fP configuration option as \fITrue\fP\&. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(options={\(aqnative_versioning\(aq: True}) .ft P .fi .UNINDENT .UNINDENT .SS Schema migrations .sp When making schema migrations (for example adding new columns to version tables) you need to remember to call sync_trigger in order to keep the version trigger up\-to\-date. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum.dialects.postgresql import sync_trigger sync_trigger(conn, \(aqarticle_version\(aq) .ft P .fi .UNINDENT .UNINDENT .sp If you don\(aqt use \fIPropertyModTrackerPlugin\fP, then you have to disable it: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C sync_trigger(conn, \(aqarticle_version\(aq, use_property_mod_tracking=False) .ft P .fi .UNINDENT .UNINDENT .SH PLUGINS .SS Using plugins .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum.plugins import PropertyModTrackerPlugin versioning_manager.plugins.append(PropertyModTrackerPlugin()) versioning_manager.plugins # # You can also remove plugin del versioning_manager.plugins[0] .ft P .fi .UNINDENT .UNINDENT .SS Activity .sp The ActivityPlugin is the most powerful plugin for tracking changes of individual entities. If you use ActivityPlugin you probably don\(aqt need to use TransactionChanges nor TransactionMeta plugins. .sp You can initalize the ActivityPlugin by adding it to versioning manager. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C activity_plugin = ActivityPlugin() make_versioned(plugins=[activity_plugin]) .ft P .fi .UNINDENT .UNINDENT .sp ActivityPlugin uses single database table for tracking activities. This table follows the data structure in \fI\%activity stream specification\fP, but it comes with a nice twist: .INDENT 0.0 .INDENT 3.5 .TS center; |l|l|l|. _ T{ Column T} T{ Type T} T{ Description T} _ T{ id T} T{ BigInteger T} T{ The primary key of the activity T} _ T{ verb T} T{ Unicode T} T{ Verb defines the action of the activity T} _ T{ data T} T{ JSON T} T{ Additional data for the activity in JSON format T} _ T{ transaction_id T} T{ BigInteger T} T{ The transaction this activity was associated with T} _ T{ object_id T} T{ BigInteger T} T{ The primary key of the object. Object can be any entity which has an integer as primary key. T} _ T{ object_type T} T{ Unicode T} T{ The type of the object (class name as string) T} _ T{ object_tx_id T} T{ BigInteger T} T{ The last transaction_id associated with the object. This is used for efficiently fetching the object version associated with this activity. T} _ T{ target_id T} T{ BigInteger T} T{ The primary key of the target. Target can be any entity which has an integer as primary key. T} _ T{ target_type T} T{ Unicode T} T{ The of the target (class name as string) T} _ T{ target_tx_id T} T{ BigInteger T} T{ The last transaction_id associated with the target. T} _ .TE .UNINDENT .UNINDENT .sp Each Activity has relationships to actor, object and target but it also holds information about the associated transaction and about the last associated transactions with the target and object. This allows each activity to also have object_version and target_version relationships for introspecting what those objects and targets were in given point in time. All these relationship properties use \fI\%generic relationships\fP of the SQLAlchemy\-Utils package. .SS Limitations .sp Currently all changes to parent models must be flushed or committed before creating activities. This is due to a fact that there is still no dependency processors for generic relationships. So when you create activities and assign objects / targets for those please remember to flush the session before creating an activity: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqSome article\(aq) session.add(article) session.flush() # <\- IMPORTANT! first_activity = Activity(verb=u\(aqcreate\(aq, object=article) session.add(first_activity) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp Targets and objects of given activity must have an integer primary key column id. .SS Create activities .sp Once your models have been configured you can get the Activity model from the ActivityPlugin class with activity_cls property: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C Activity = activity_plugin.activity_cls .ft P .fi .UNINDENT .UNINDENT .sp Now let\(aqs say we have model called Article and Category. Each Article has one Category. Activities should be created along with the changes you make on these models. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqSome article\(aq) session.add(article) session.flush() first_activity = Activity(verb=u\(aqcreate\(aq, object=article) session.add(first_activity) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp Current transaction gets automatically assigned to activity object: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C first_activity.transaction # Transaction object .ft P .fi .UNINDENT .UNINDENT .SS Update activities .sp The object property of the Activity object holds the current object and the object_version holds the object version at the time when the activity was created. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C article.name = u\(aqSome article updated!\(aq session.flush() second_activity = Activity(verb=u\(aqupdate\(aq, object=article) session.add(second_activity) session.commit() second_activity.object.name # u\(aqSome article updated!\(aq first_activity.object.name # u\(aqSome article updated!\(aq first_activity.object_version.name # u\(aqSome article\(aq .ft P .fi .UNINDENT .UNINDENT .SS Delete activities .sp The version properties are especially useful for delete activities. Once the activity is fetched from the database the object is no longer available ( since its deleted), hence the only way we could show some information about the object the user deleted is by accessing the object_version property. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C session.delete(article) session.flush() third_activity = Activity(verb=u\(aqdelete\(aq, object=article) session.add(third_activity) session.commit() third_activity.object_version.name # u\(aqSome article updated!\(aq .ft P .fi .UNINDENT .UNINDENT .SS Local version histories using targets .sp The target property of the Activity model offers a way of tracking changes of given related object. In the example below we create a new activity when adding a category for article and then mark the article as the target of this activity. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C session.add(Category(name=u\(aqFist category\(aq, article=article)) session.flush() activity = Activity( verb=u\(aqcreate\(aq, object=category, target=article ) session.add(activity) session.commit() .ft P .fi .UNINDENT .UNINDENT .sp Now if we wanted to find all the changes that affected given article we could do so by searching through all the activities where either the object or target is the given article. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C import sqlalchemy as sa activities = session.query(Activity).filter( sa.or_( Activity.object == article, Activity.target == article ) ) .ft P .fi .UNINDENT .UNINDENT .SS Flask .sp FlaskPlugin offers way of integrating Flask framework with SQLAlchemy\-Continuum. Flask\-Plugin adds two columns for Transaction model, namely \fIuser_id\fP and \fIremote_addr\fP\&. .sp These columns are automatically populated when transaction object is created. The \fIremote_addr\fP column is populated with the value of the remote address that made current request. The \fIuser_id\fP column is populated with the id of the current_user object. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum.plugins import FlaskPlugin from sqlalchemy_continuum import make_versioned make_versioned(plugins=[FlaskPlugin()]) .ft P .fi .UNINDENT .UNINDENT .SS PropertyModTracker .sp The PropertyModTrackerPlugin offers a way of efficiently tracking individual property modifications. With PropertyModTrackerPlugin you can make efficient queries such as: .sp Find all versions of model X where user updated the property A or property B. .sp Find all versions of model X where user didn\(aqt update property A. .sp PropertyModTrackerPlugin adds separate modified tracking column for each versioned column. So for example if you have versioned model called Article with columns \fIname\fP and \fIcontent\fP, this plugin would add two additional boolean columns \fIname_mod\fP and \fIcontent_mod\fP for the version model. When user commits transactions the plugin automatically updates these boolean columns. .SS TransactionChanges .sp TransactionChanges provides way of keeping track efficiently which declarative models were changed in given transaction. This can be useful when transactions need to be queried afterwards for problems such as: .INDENT 0.0 .IP 1. 3 Find all transactions which affected \fIUser\fP model. .IP 2. 3 Find all transactions which didn\(aqt affect models \fIEntity\fP and \fIEvent\fP\&. .UNINDENT .sp The plugin works in two ways. On class instrumentation phase this plugin creates a special transaction model called \fITransactionChanges\fP\&. This model is associated with table called \fItransaction_changes\fP, which has only only two fields: transaction_id and entity_name. If for example transaction consisted of saving 5 new User entities and 1 Article entity, two new rows would be inserted into transaction_changes table. .TS center; |l|l|. _ T{ transaction_id T} T{ entity_name T} _ T{ 233678 T} T{ User T} _ T{ 233678 T} T{ Article T} _ .TE .SS TransactionMeta .sp TransactionMetaPlugin offers a way of saving key\-value data for transations. You can use the plugin in same way as other plugins: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C meta_plugin = TransactionMetaPlugin() versioning_manager.plugins.add(meta_plugin) .ft P .fi .UNINDENT .UNINDENT .sp TransactionMetaPlugin creates a simple model called TransactionMeta. This class has three columns: transaction_id, key and value. TransactionMeta plugin also creates an association proxy between TransactionMeta and Transaction classes for easy dictionary based access of key\-value pairs. .sp You can easily \(aqtag\(aq transactions with certain key value pairs by giving these keys and values to the meta property of Transaction class. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import versioning_manager article = Article() session.add(article) uow = versioning_manager.unit_of_work(session) tx = uow.create_transaction(session) tx.meta = {u\(aqsome_key\(aq: u\(aqsome value\(aq} session.commit() TransactionMeta = meta_plugin.model_class Transaction = versioning_manager.transaction_cls # find all transactions with \(aqarticle\(aq tags query = ( session.query(Transaction) .join(Transaction.meta_relation) .filter( db.and_( TransactionMeta.key == \(aqsome_key\(aq, TransactionMeta.value == \(aqsome value\(aq ) ) ) .ft P .fi .UNINDENT .UNINDENT .SH CONFIGURATION .SS Global and class level configuration .sp All Continuum configuration parameters can be set on global level (manager level) and on class level. Setting an option at manager level affects all classes within the scope of the manager\(aqs class instrumentation listener (by default all SQLAlchemy declarative models). .sp In the following example we set \(aqtransaction_column_name\(aq configuration option to False at the manager level. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(options={\(aqtransaction_column_name\(aq: \(aqmy_tx_id\(aq}) .ft P .fi .UNINDENT .UNINDENT .sp As the name suggests class level configuration only applies to given class. Class level configuration can be passed to __versioned__ class attribute. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class User(Base): __versioned__ = { \(aqtransaction_column_name\(aq: \(aqtx_id\(aq } .ft P .fi .UNINDENT .UNINDENT .SS Versioning strategies .sp Similar to Hibernate Envers SQLAlchemy\-Continuum offers two distinct versioning strategies \(aqvalidity\(aq and \(aqsubquery\(aq. The default strategy is \(aqvalidity\(aq. .SS Validity .sp The \(aqvalidity\(aq strategy saves two columns in each history table, namely \(aqtransaction_id\(aq and \(aqend_transaction_id\(aq. The names of these columns can be configured with configuration options \fItransaction_column_name\fP and \fIend_transaction_column_name\fP\&. .sp As with \(aqsubquery\(aq strategy for each inserted, updated and deleted entity Continuum creates new version in the history table. However it also updates the end_transaction_id of the previous version to point at the current version. This creates a little bit of overhead during data manipulation. .sp With \(aqvalidity\(aq strategy version traversal is very fast. When accessing previous version Continuum tries to find the version record where the primary keys match and end_transaction_id is the same as the transaction_id of the given version record. When accessing the next version Continuum tries to find the version record where the primary keys match and transaction_id is the same as the end_transaction_id of the given version record. .INDENT 0.0 .TP .B Pros: .INDENT 7.0 .IP \(bu 2 Version traversal is much faster since no correlated subqueries are needed .UNINDENT .TP .B Cons: .INDENT 7.0 .IP \(bu 2 Updates, inserts and deletes are little bit slower .UNINDENT .UNINDENT .SS Subquery .sp The \(aqsubquery\(aq strategy uses one column in each history table, namely \(aqtransaction_id\(aq. The name of this column can be configured with configuration option \fItransaction_column_name\fP\&. .sp After each inserted, updated and deleted entity Continuum creates new version in the history table and sets the \(aqtransaction_id\(aq column to point at the current transaction. .sp With \(aqsubquery\(aq strategy the version traversal is slow. When accessing previous and next versions of given version object needs correlated subqueries. .INDENT 0.0 .TP .B Pros: .INDENT 7.0 .IP \(bu 2 Updates, inserts and deletes little bit faster than in \(aqvalidity\(aq strategy .UNINDENT .TP .B Cons: .INDENT 7.0 .IP \(bu 2 Version traversel much slower .UNINDENT .UNINDENT .SS Column exclusion and inclusion .sp With \fIexclude\fP configuration option you can define which entity attributes you want to get versioned. By default Continuum versions all entity attributes. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class User(Base): __versioned__ = { \(aqexclude\(aq: [\(aqpicture\(aq] } id = sa.Column(sa.Integer, primary_key=True) name = sa.Column(sa.Unicode(255)) picture = sa.Column(sa.LargeBinary) .ft P .fi .UNINDENT .UNINDENT .SS Basic configuration options .sp Here is a full list of configuration options: .INDENT 0.0 .IP \(bu 2 .INDENT 2.0 .TP .B base_classes (default: None) A tuple defining history class base classes. .UNINDENT .IP \(bu 2 .INDENT 2.0 .TP .B table_name (default: \(aq%s_version\(aq) The name of the history table. .UNINDENT .IP \(bu 2 .INDENT 2.0 .TP .B transaction_column_name (default: \(aqtransaction_id\(aq) The name of the transaction column (used by history tables). .UNINDENT .IP \(bu 2 .INDENT 2.0 .TP .B end_transaction_column_name (default: \(aqend_transaction_id\(aq) The name of the end transaction column in history table when using the validity versioning strategy. .UNINDENT .IP \(bu 2 .INDENT 2.0 .TP .B operation_type_column_name (default: \(aqoperation_type\(aq) The name of the operation type column (used by history tables). .UNINDENT .IP \(bu 2 .INDENT 2.0 .TP .B strategy (default: \(aqvalidity\(aq) The versioning strategy to use. Either \(aqvalidity\(aq or \(aqsubquery\(aq .UNINDENT .UNINDENT .sp Example .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C class Article(Base): __versioned__ = { \(aqtransaction_column_name\(aq: \(aqtx_id\(aq } __tablename__ = \(aquser\(aq id = sa.Column(sa.Integer, primary_key=True, autoincrement=True) name = sa.Column(sa.Unicode(255)) content = sa.Column(sa.UnicodeText) .ft P .fi .UNINDENT .UNINDENT .SS Customizing transaction user class .sp By default Continuum tries to build a relationship between \(aqUser\(aq class and Transaction class. If you have differently named user class you can simply pass its name to make_versioned: .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(user_cls=\(aqMyUserClass\(aq) .ft P .fi .UNINDENT .UNINDENT .sp If you don\(aqt want transactions to contain any user references you can also disable this feature. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(user_cls=None) .ft P .fi .UNINDENT .UNINDENT .SS Customizing versioned mappers .sp By default SQLAlchemy\-Continuum versions all mappers. You can override this behaviour by passing the desired mapper class/object to make_versioned function. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(mapper=my_mapper) .ft P .fi .UNINDENT .UNINDENT .SS Customizing versioned sessions .sp By default SQLAlchemy\-Continuum versions all sessions. You can override this behaviour by passing the desired session class/object to make_versioned function. .INDENT 0.0 .INDENT 3.5 .sp .nf .ft C make_versioned(session=my_session) .ft P .fi .UNINDENT .UNINDENT .SH CONTINUUM SCHEMA .SS Version tables .sp By default SQLAlchemy\-Continuum creates a version table for each versioned entity table. The version tables are suffixed with \(aq_version\(aq. So for example if you have two versioned tables \(aqarticle\(aq and \(aqcategory\(aq, SQLAlchemy\-Continuum would create two version tables \(aqarticle_version\(aq and \(aqcategory_version\(aq. .sp By default the version tables contain these columns: .INDENT 0.0 .IP \(bu 2 id of the original entity (this can be more then one column in the case of composite primary keys) .IP \(bu 2 transaction_id \- an integer that matches to the id number in the transaction_log table. .IP \(bu 2 end_transaction_id \- an integer that matches the next version record\(aqs transaction_id. If this is the current version record then this field is null. .IP \(bu 2 operation_type \- a small integer defining the type of the operation .IP \(bu 2 versioned fields from the original entity .UNINDENT .sp If you are using \fI\%PropertyModTracker\fP Continuum also creates one boolean field for each versioned field. By default these boolean fields are suffixed with \(aq_mod\(aq. .sp The primary key of each version table is the combination of parent table\(aqs primary key + the transaction_id. This means there can be at most one version table entry for a given entity instance at given transaction. .SS Transaction tables .sp By default Continuum creates one transaction table called \fItransaction\fP\&. Many continuum plugins also create additional tables for efficient transaction storage. If you wish to query efficiently transactions afterwards you should consider using some of these plugins. .sp The transaction table only contains two fields by default: id and issued_at. .SS Using vacuum .INDENT 0.0 .TP .B sqlalchemy_continuum.vacuum(session, model, yield_per=1000) When making structural changes to version tables (for example dropping columns) there are sometimes situations where some old version records become futile. .sp Vacuum deletes all futile version rows which had no changes compared to previous version. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import vacuum vacuum(session, User) # vacuums user version .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters .INDENT 7.0 .IP \(bu 2 \fBsession\fP \-\- SQLAlchemy session object .IP \(bu 2 \fBmodel\fP \-\- SQLAlchemy declarative model class .IP \(bu 2 \fByield_per\fP \-\- how many rows to process at a time .UNINDENT .UNINDENT .UNINDENT .SS Schema tools .INDENT 0.0 .TP .B sqlalchemy_continuum.schema.update_end_tx_column(table, end_tx_column_name=\(aqend_transaction_id\(aq, tx_column_name=\(aqtransaction_id\(aq, conn=None) Calculates end transaction columns and updates the version table with the calculated values. This function can be used for migrating between subquery versioning strategy and validity versioning strategy. .INDENT 7.0 .TP .B Parameters .INDENT 7.0 .IP \(bu 2 \fBtable\fP \-\- SQLAlchemy table object .IP \(bu 2 \fBend_tx_column_name\fP \-\- Name of the end transaction column .IP \(bu 2 \fBtx_column_name\fP \-\- Transaction column name .IP \(bu 2 \fBconn\fP \-\- .sp Either SQLAlchemy Connection, Engine, Session or Alembic Operations object. Basically this should be an object that can execute the queries needed to update the end transaction column values. .sp If no object is given then this function tries to use alembic.op for executing the queries. .UNINDENT .UNINDENT .UNINDENT .INDENT 0.0 .TP .B sqlalchemy_continuum.schema.update_property_mod_flags(table, tracked_columns, mod_suffix=\(aq_mod\(aq, end_tx_column_name=\(aqend_transaction_id\(aq, tx_column_name=\(aqtransaction_id\(aq, conn=None) Update property modification flags for given table and given columns. This function can be used for migrating an existing schema to use property mod flags (provided by PropertyModTracker plugin). .INDENT 7.0 .TP .B Parameters .INDENT 7.0 .IP \(bu 2 \fBtable\fP \-\- SQLAlchemy table object .IP \(bu 2 \fBmod_suffix\fP \-\- Modification tracking columns suffix .IP \(bu 2 \fBend_tx_column_name\fP \-\- Name of the end transaction column .IP \(bu 2 \fBtx_column_name\fP \-\- Transaction column name .IP \(bu 2 \fBconn\fP \-\- .sp Either SQLAlchemy Connection, Engine, Session or Alembic Operations object. Basically this should be an object that can execute the queries needed to update the property modification flags. .sp If no object is given then this function tries to use alembic.op for executing the queries. .UNINDENT .UNINDENT .UNINDENT .SH ALEMBIC MIGRATIONS .sp Each time you make changes to database structure you should also change the associated history tables. When you make changes to your models SQLAlchemy\-Continuum automatically alters the history model definitions, hence you can use \fIalembic revision \-\-autogenerate\fP just like before. You just need to make sure \fImake_versioned\fP function gets called before alembic gathers all your models and \fIconfigure_mappers\fP is called afterwards. .sp Pay close attention when dropping or moving data from parent tables and reflecting these changes to history tables. .SS Troubleshooting .sp If alembic didn\(aqt detect any changes or generates reversed migration (tries to remove \fI*_version\fP tables from database instead of creating), make sure that \fIconfigure_mappers\fP was called by alembic command. .SH UTILITIES .SS changeset .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.changeset(obj) Return a humanized changeset for given SQLAlchemy declarative object. With this function you can easily check the changeset of given object in current transaction. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import changeset article = Article(name=u\(aqSome article\(aq) changeset(article) # {\(aqname\(aq: [u\(aqSome article\(aq, None]} .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBobj\fP \-\- SQLAlchemy declarative model object .UNINDENT .UNINDENT .SS count_versions .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.count_versions(obj) Return the number of versions given object has. This function works even when obj has \fIcreate_models\fP and \fIcreate_tables\fP versioned settings disabled. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C article = Article(name=u\(aqSome article\(aq) count_versions(article) # 0 session.add(article) session.commit() count_versions(article) # 1 .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBobj\fP \-\- SQLAlchemy declarative model object .UNINDENT .UNINDENT .SS get_versioning_manager .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.get_versioning_manager(obj_or_class) Return the associated SQLAlchemy\-Continuum VersioningManager for given SQLAlchemy declarative model class or object. .INDENT 7.0 .TP .B Parameters \fBobj_or_class\fP \-\- SQLAlchemy declarative model object or class .UNINDENT .UNINDENT .SS is_modified .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.is_modified(obj) Return whether or not the versioned properties of given object have been modified. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C article = Article() is_modified(article) # False article.name = \(aqSomething\(aq is_modified(article) # True .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBobj\fP \-\- SQLAlchemy declarative model object .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%is_modified_or_deleted()\fP .UNINDENT .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%is_session_modified()\fP .UNINDENT .UNINDENT .UNINDENT .SS is_modified_or_deleted .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.is_modified_or_deleted(obj) Return whether or not some of the versioned properties of given SQLAlchemy declarative object have been modified or if the object has been deleted. .INDENT 7.0 .TP .B Parameters \fBobj\fP \-\- SQLAlchemy declarative model object .UNINDENT .UNINDENT .SS is_session_modified .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.is_session_modified(session) Return whether or not any of the versioned objects in given session have been either modified or deleted. .INDENT 7.0 .TP .B Parameters \fBsession\fP \-\- SQLAlchemy session object .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%is_versioned()\fP .UNINDENT .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%versioned_objects()\fP .UNINDENT .UNINDENT .UNINDENT .SS is_versioned .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.is_versioned(obj_or_class) Return whether or not given object is versioned. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C is_versioned(Article) # True article = Article() is_versioned(article) # True .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBobj_or_class\fP \-\- SQLAlchemy declarative model object or SQLAlchemy declarative model class. .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%versioned_objects()\fP .UNINDENT .UNINDENT .UNINDENT .SS parent_class .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.parent_class(version_cls) Return the parent class for given version model class. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C parent_class(ArticleVersion) # Article class .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBmodel\fP \-\- SQLAlchemy declarative version model class .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%version_class()\fP .UNINDENT .UNINDENT .UNINDENT .SS transaction_class .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.transaction_class(cls) Return the associated transaction class for given versioned SQLAlchemy declarative class or version class. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C from sqlalchemy_continuum import transaction_class transaction_class(Article) # Transaction class .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBcls\fP \-\- SQLAlchemy versioned declarative class or version model class .UNINDENT .UNINDENT .SS version_class .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.version_class(model) Return the version class for given SQLAlchemy declarative model class. .INDENT 7.0 .INDENT 3.5 .sp .nf .ft C version_class(Article) # ArticleVersion class .ft P .fi .UNINDENT .UNINDENT .INDENT 7.0 .TP .B Parameters \fBmodel\fP \-\- SQLAlchemy declarative model class .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%parent_class()\fP .UNINDENT .UNINDENT .UNINDENT .SS versioned_objects .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.versioned_objects(session) Return all versioned objects in given session. .INDENT 7.0 .TP .B Parameters \fBsession\fP \-\- SQLAlchemy session object .UNINDENT .sp \fBSEE ALSO:\fP .INDENT 7.0 .INDENT 3.5 \fI\%is_versioned()\fP .UNINDENT .UNINDENT .UNINDENT .SS version_table .INDENT 0.0 .TP .B sqlalchemy_continuum.utils.version_table(table) Return associated version table for given SQLAlchemy Table object. .INDENT 7.0 .TP .B Parameters \fBtable\fP \-\- SQLAlchemy Table object .UNINDENT .UNINDENT .SH LICENSE .sp Copyright (c) 2012, Konsta Vesterinen .sp All rights reserved. .sp Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: .INDENT 0.0 .IP \(bu 2 Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. .IP \(bu 2 Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. .IP \(bu 2 Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. .UNINDENT .sp THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \(dqAS IS\(dq AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH AUTHOR Konsta Vesterinen .SH COPYRIGHT 2024, Konsta Vesterinen .\" Generated by docutils manpage writer. .