Imagine we have a database table of customer details which sits behind a typical web application:
|1||Ben Smith||33||Male||1 High Street, London|
|2||Sue Jones||35||Female||1 Low Street, Manchester|
The majority of screens in the majority of business applications allow direct changes to this table, for instance by overwriting the data with updates, or deleting rows.
When you click Save on a web form, perhaps some SQL is executed on the database such as UPDATE NAME = “Sue Wilson” WHERE ID 2. The customers old name “Sue Jones” is essentially lost to the depths of the database transaction logs and your application log files.
Statements like this run trillions of times per day across the world, and each time a great deal of useful information is lost.
This information is valuable for lots of operational reasons, but especially for audit and compliance, where we need to do know exactly what data changed, when, why, and by whom.
Where this type of information clearly needs to be maintained, this will often explicitly be added using techniques such as Type 4 Slowly Changing Dimension tables which maintain a history table with timestamps. Information such as the username of the operator who entered the new information could also be logged here.
|1||Ben Smith||33||Male||1 High Street, London||1st Jan 2012|
|2||Sue Jones||35||Female||1 Low Street, Manchester||1st Jan 2019|
|2||Sue Wilson||35||Female||1 Low Street, Manchester||3rd Feb 2020|
The problem is that this requires recognising that we need to maintain history and then explicitly coding for it. Slowly Changing Dimensions are not an inherent property of our architecture or database, so by default we will be losing data on each transaction – which is a compliance nightmare.
Event Driven Architectures consist of loosely coupled services that respond to real world events. In the above example, an event could be Name Changed, and all of the services which need to respond to that event will subscribe to it and respond accordingly, updating state and triggering necessary actions. If this were a bank for instance, we might wish to save the new name and re-issue the debit card in response to the event.
For various reasons, modern “Event Driven” architectures are inherently better at solving the “lost data” problem than traditional CRUD architectures.
As mentioned, it’s perfectly possible to lose information by accepting events and translating them back to destructive CRUD updates, but because of the additional transaction logs, the coding styles and the way that event based architectures persist data, we start off in a much stronger position for audit and compliance in relation to how our data was built up over time.