Valverde

Computing

 The Open Source/ Systems Mainframe Solution

The Fundamentals of Transaction Systems Part 1:
Causality banishes Acausality
in Clustered Database

Video: Windows | Flash First Hour:

• The "Elevator Pitch"
• a brief exposure to the optimal clustering software and hardware architecture
• the Four Pillars of clustered systems design
• Reliable Message Based System - serialized retries with duplicate removal

Video: Windows | Flash Second Hour:

• Data Integrity - data must be checked wherever it goes
• Reliability = fail-fast + fault detection + fault tolerance + fault avoidance + proper fault containment
• Basic Parallelism - if it isn’t locked, then it isn’t blocked
• Basic Transparency - when? where? how?
• Basic Scalability

Video: Windows | Flash Third Hour:

• Basic Scalability (continued):
• Designing the streaming TM-RM-Log machinery, with the state machine in the log root
• Basic Availability - outage minutes -> zero
• Application/Database Serialized Consistency - the database must be serialized wherever it goes:
• Implementing ACID properties
• The log is the database
• MVCC vs. S2PL
• Recovery - putting it all back together again, minimizing recovery times (MTR)‏

The Fundamentals of Transaction Systems Part 2:
Certainty suppresses Uncertainty
in Groups of Clusters

Video: Windows | Flash Fourth Hour:

• ACID and BASE - workflow makes this reaction safe
• True Multi-Threading - shrinking the size of thread-instance state
• Single System Image and Network Autonomy
• Minimal Use of Special Hardware - servers need to be off-the-shelf
• Maintainability and Supportability - H/W & S/W needs to be capable of basic on-line repair
• Expansively Transparent Parallelism and Scalability

Video: Windows | Flash Fifth Hour:

• Expansively Transparent Parallelism and Scalability (continued)
• Continuous Database - needs virtual commit by name:
• Designing virtual commit coordination for massive availability in datacenters
• Log storage groups a la Nonstop

The Fundamentals of Transaction Systems Part 3:
Relativity shatters the Classical Delusion
in Replicated Database

Video: Windows | Flash  Sixth Hour:

• Reliable Disjoint Async Replication in non-distributed systems:
• Some definitions
• A current survey of critical systems disjoint replication products
• Some augmentations to those and a new direction
• Logical Redo and Volume Autonomy:
• Some theory
• Some practice
• A new direction
•  Scalable Joint Replication in distributed systems:
• Some definitions
• A current survey of critical systems joint replication products
• Some augmentations to those

Video: Windows | Flash Seventh Hour:

• Scalable Joint Replication (continued):
• A demonstration application for the geographically-dispersed replication of a distributed database is considered: a redesign of the Deep Space Network
• The design of a combined 1-Safe/2-Safe/Very-Safe system begins with 1-Safe (the difficult, racy method):
• The 1-safe issues
• A major constraint
• The approach taken with various log position pointers
• The state machine in the log root
• And a state machine in the log partitions for unhindered dispersion and quick transaction resolution to limit MTR
• Distributed low (undo and redo) and high water marks

Video: Windows | Flash Eighth Hour:

• Scalable Joint Replication (continued):
• Completing the 1-Safe apparatus:
• Three (reasonable) consistency policies
• Takeover processing
• Bringing the dispersed database back online in seconds
• Final resolution of the last transactions and online rollback
• Reintegration and giveover to return to the initial state of the system
• Starting the 2-safe /very-safe design

Video: Windows | Flash Ninth Hour:

• Scalable Joint Replication (completing):
• Completing the 2-safe /very-safe design:
• How 2-safe and very-safe are alike and how different
• The hybrid option
• The 2-safe/very-safe issues
• The approach taken with various log position pointers
• The more complex (than 1-safe) state machine in the log root
• And the far more complex (than 1-safe) state machine in the log partitions for unhindered dispersion and quick transaction resolution to limit MTR
• Bringing 1-safe/2-safe/very-safe and the hybrid option together happily co-existing in the same log root
• Finally completing the redesign of the Deep Space Network
• (TBD) Bi-Directional Replication - Reliable, Scalable, Atomically Consistent
  

The Fundamentals of Transaction Systems Part 4:
Purity emerges from Impurity
in Practical makes perfect

Video: Windows | Flash  Tenth Hour:

• This razor needs razor blades: an open source set of applications creating a virtual operating system on top of the groups of clusters, connected by replication streams, which are joined by workflow systems:
• Openness (Glasnost) - Open systems, open source, free software
• Restructuring (Perestroika) - Online application and schema maintenance
• Reliable Software Telemetry - push streaming needs a many-to-many architecture
  
• (TBD) Publish and Subscribe
• (TBD) Ubiquitous Work Flow
• (TBD) Virtual Operating System
• Scaling Inwards - Extreme Single Row Performance for Exchanges
• (TBD) Ad Hoc Aggregation - Institutional Query Transparency for Regulation
• (TBD) Reliable Multi-Lateral Trading - Regulated Fairness & Performance, Guaranteed Result
• (TBD) Semantic Data - Verity of Data Processing
• (TBD) Integration and Test Platform - Real-Time Transaction Database
• (TBD) Integrated Logistics
  

Click below for direct file access:

(Right-click to download and save an hour of video to your system, they are free to use and distribute)