ABCDeez Documentation

Welcome to the ABCDeez documentation. This comprehensive guide covers the adaptive learning system's architecture, implementation details, and behavioral specifications.

Overview

ABCDeez is an adaptive learning system designed for research and educational purposes. It implements sophisticated learning algorithms, experimental protocols, and comprehensive data collection capabilities.

Documentation Structure

This documentation is organized into three main sections:

Core Module

The foundational learning system implementation, including:

Adaptive learning algorithms
Task definitions and boundaries
Statistical analysis framework
Experimental design protocols
Compliance and audit systems

Web Backend

The server-side implementation providing:

RESTful API endpoints
WebSocket real-time communication
Database and caching layers
Authentication and authorization
Monitoring and observability

Xilem Application

The cross-platform client application featuring:

Native UI components
Real-time synchronization
Offline capabilities
Adaptive learning interfaces
Task visualization

Key Features

Adaptive Learning: Bayesian and hierarchical Bayesian models for personalized learning paths
Experimental Protocols: Support for A/B testing, multi-session studies, and preregistration
Compliance: Built-in IRB compliance, audit trails, and citation management
Real-time Sync: WebSocket-based synchronization across devices
Cross-platform: Support for web, desktop, and mobile platforms
Observability: Comprehensive monitoring with OpenTelemetry integration

Getting Started

For a quick introduction to using ABCDeez, see the Core Module documentation.

For API integration, refer to the Web Backend documentation.

For client application development, check the Xilem App documentation.

lib.rs - Library Root Module Abstract

High-Level Purpose

The library root module serves as the public API gateway for the abcdeez-core adaptive learning research framework. It defines the organizational structure and provides unified access to all functional domains of the system.

Key Data Structures and Relationships

Module Organization: Hierarchical module structure exposing eight primary domains:
- compliance: Research compliance and regulatory requirements
- core: Fundamental system architecture and error handling
- data: Data collection, processing, and export capabilities
- demo: Demonstration and example implementations
- experiments: Experimental design and execution framework
- learning: Adaptive learning algorithms and strategies
- protocol: Communication protocols and version management
- statistics: Statistical analysis and validation tools
- tasks: Task definition and execution framework

Main Data Flows

Public API Exposure: Re-exports core error types (Error, Result) for unified error handling across the framework
Version Management: Exposes package version information at compile time
Prelude Pattern: Provides a convenience module for common imports

External Dependencies

Standard Rust module system
Cargo build system for version information (env! macro)
Conditional compilation for test modules

State Management Patterns

Stateless Module Organization: Pure module aggregation without internal state
Error Propagation: Centralized error type system through re-exports

Core Abstractions

Domain Separation: Clear separation of concerns across functional domains
API Consistency: Unified error handling and result types across all modules
Build-time Configuration: Compile-time version embedding for runtime access

core/backend.rs - Backend Communication System Abstract

High-Level Purpose

Comprehensive backend communication system enabling reliable data synchronization, participant management, and experiment coordination with external research infrastructure, supporting both synchronous and asynchronous operation modes.

Key Data Structures and Relationships

BackendClient: Primary communication interface with retry logic and buffering
BackendConfig: Flexible configuration supporting environment and file-based setup
SessionToken: Secure participant session management with expiration handling
ResponseBatch: Efficient batch data transmission with automatic buffering
ExperimentConfig: Remote experiment configuration retrieval and management

Main Data Flows

Participant Registration: Secure participant enrollment with group assignment
Session Management: Session lifecycle management with token-based authentication
Data Buffering: Automatic response buffering with configurable batch sizes
Batch Synchronization: Reliable data transmission with exponential backoff retry
Real-time Metrics: Live performance metrics transmission for monitoring

External Dependencies

reqwest: HTTP client for backend communication (both blocking and async variants)
anyhow: Error handling and context management
chrono: Timestamp management for session and data tracking
serde: Serialization for API communication
tokio: Asynchronous operation support (optional async feature)

State Management Patterns

Connection State: Persistent connection management with health checking
Buffer Management: Response buffering with automatic overflow handling
Session State: Secure session tracking with token lifecycle management
Sync State: Last synchronization tracking for efficient data transmission

Core Algorithms and Business Logic Abstractions

Retry Strategy: Exponential backoff with configurable retry limits
Batch Optimization: Intelligent batching based on size and time thresholds
Health Monitoring: Connection health checking and error reporting
Data Integrity: Reliable data transmission with error detection and recovery
Authentication: Token-based authentication with automatic renewal

Communication Patterns

RESTful API: Standard HTTP REST API communication patterns
Batch Processing: Efficient bulk data transmission for performance optimization
Real-time Updates: Live metrics and status updates for experiment monitoring
Error Reporting: Comprehensive error reporting and diagnostic information
Configuration Sync: Remote configuration retrieval and validation

Reliability Features

Connection Resilience: Automatic reconnection and retry mechanisms
Data Persistence: Local buffering with guaranteed delivery semantics
Error Recovery: Graceful degradation and recovery from communication failures
Timeout Handling: Configurable timeouts with appropriate fallback behavior
Background Sync: Asynchronous background synchronization for optimal performance

core/config.rs - System Configuration Management Abstract

High-Level Purpose

Comprehensive configuration management system providing validated, hierarchical configuration for all framework components, with population-specific presets and domain-specific parameter tuning for adaptive learning systems.

Key Data Structures and Relationships

SystemConfig: Top-level configuration aggregating all subsystem configurations
LearnerConfig: Individual learner behavior parameters and learning dynamics
AdaptiveSchedulingConfig: Task scheduling and adaptation parameters
HintInterventionConfig: Intelligent hint system configuration
DomainConfig: Domain-specific task difficulties and knowledge structures
Preset System: Population and goal-specific configuration templates

Main Data Flows

Configuration Validation: Comprehensive parameter validation with range and consistency checking
Preset Selection: Population-type and domain-specific configuration generation
Parameter Hierarchies: Nested configuration structures with inheritance and overrides
Serialization Support: JSON/TOML serialization for persistent configuration storage

External Dependencies

serde: Configuration serialization and deserialization
Standard library collections for configuration data structures

State Management Patterns

Immutable Configuration: Read-only configuration objects with validation
Preset Templates: Pre-defined configuration templates for common scenarios
Validation State: Comprehensive validation with detailed error reporting

Core Algorithms and Business Logic Abstractions

Parameter Validation: Multi-level validation with domain-specific constraints
Population Modeling: Evidence-based parameter sets for different learner populations
Domain Adaptation: Task difficulty and response time modeling for different knowledge domains
Adaptive Tuning: Dynamic parameter adjustment based on learning goals and population characteristics

Configuration Domains

Learning Parameters: Memory strength, uncertainty, proficiency modeling
Adaptive Scheduling: Epsilon-greedy exploration, success rate targeting, scoring weights
Hint Systems: Struggle detection, error streak thresholds, adaptive timing
Domain Knowledge: Task difficulties, chunk boundaries, response time baselines
Population Presets: Adult, child, older adult, learning disability, and expert configurations

Validation and Quality Assurance

Range Validation: Parameter bounds checking with domain-appropriate limits
Consistency Validation: Cross-parameter consistency and logical constraint checking
Weight Normalization: Automatic validation of probability distributions and weight sums
Evidence-Based Defaults: Scientifically-grounded default parameters based on research literature

core/topology.rs - Knowledge Structure Topology System Abstract

High-Level Purpose

Comprehensive graph-based knowledge representation system supporting multiple topology types (linear, cyclic, DAG, general graphs) for modeling structured learning domains with sophisticated path finding and relationship analysis capabilities.

Key Data Structures and Relationships

Topology: Central graph structure with type-specific optimizations
Node: Knowledge elements with labels, positions, and unique identifiers
Edge: Directed relationships with weighted connections between knowledge elements
TopologyType: Enumerated topology variants with specialized algorithms
Node Mapping: Efficient lookup structures for label-to-index resolution

Main Data Flows

Graph Construction: Type-specific graph building with validation and optimization
Path Finding: Dijkstra's algorithm for shortest path computation in weighted graphs
Relationship Queries: Successor/predecessor relationships and distance calculations
Traversal Operations: Topological sorting, segment extraction, and path enumeration
Comparative Analysis: Node ordering and relationship comparisons across topology types

External Dependencies

serde: Serialization support for graph persistence and transmission
std::collections: HashMap, HashSet, VecDeque for efficient graph algorithms

State Management Patterns

Immutable Structures: Read-only graph structures after construction
Efficient Indexing: Pre-computed node mappings for O(1) lookup operations
Type-Specific Optimization: Specialized algorithms for different topology types

Core Algorithms and Business Logic Abstractions

Graph Algorithms: Dijkstra's shortest path, topological sorting, BFS traversal
Distance Metrics: Type-specific distance calculations (linear, cyclic, graph-based)
Relationship Analysis: Path existence, comparability, and ordering relationships
Segment Operations: Sequential element extraction with directional support
Validation Logic: Graph structure validation and consistency checking

Topology Types and Applications

Linear Topology: Sequential structures like alphabets and ordered lists
Cyclic Topology: Circular structures like days of week and periodic sequences
Partial Order (DAG): Hierarchical dependencies and prerequisite relationships
General Graph: Arbitrary weighted networks with complex interconnections

Knowledge Domain Support

Alphabet Learning: 26-letter linear structure with position-based operations
Temporal Sequences: Cyclic structures for time-based learning
Skill Dependencies: DAG structures for prerequisite learning paths
Arbitrary Domains: Flexible graph structures for complex knowledge representations

Performance Optimizations

Pre-computed Indices: O(1) node lookup through hash-based indexing
Type-Specific Algorithms: Optimized algorithms for each topology type
Memory Efficiency: Compact representation with minimal overhead
Algorithm Selection: Automatic algorithm selection based on topology type

core/error.rs - Unified Error Handling System Abstract

High-Level Purpose

Comprehensive error handling system providing structured error types and unified result handling for all framework operations, enabling consistent error propagation and debugging across the abcdeez-core system.

Key Data Structures and Relationships

Error Enum: Comprehensive error categorization covering all framework domains
Result Type Alias: Standardized Result<T, Error> for consistent error handling
Error Variants: Domain-specific error types with contextual information
Source Chain: Error chaining for root cause analysis

Main Data Flows

Error Creation: Structured error creation with contextual information
Error Propagation: Seamless error propagation through ? operator
Error Conversion: Automatic conversion from standard library and external errors
Error Display: Human-readable error messages with detailed context

External Dependencies

std::error: Standard library error trait implementation
std::fmt: Display formatting for error messages
serde_json: Serialization error handling

State Management Patterns

Stateless Errors: Immutable error types with captured context
Error Context: Rich contextual information including parameters and state

Core Algorithms and Business Logic Abstractions

Error Classification: Systematic categorization of failure modes
Context Preservation: Detailed error context for debugging and recovery
Domain-Specific Errors: Specialized error types for different system components
Recovery Information: Structured error data enabling automated recovery strategies

Error Categories

Topology Errors: Invalid graph structures and operations
Task Generation: Task creation and validation failures
Numerical Errors: Mathematical computation failures
Parameter Validation: Invalid configuration and parameter errors
Data Sufficiency: Insufficient data for statistical operations
Convergence Issues: Algorithm convergence failures with diagnostic information

Learning Module - Abstract Documentation

Purpose and Responsibility

Implements the core adaptive learning algorithms including Bayesian updating, strategy mixture models, hierarchical learning, and transfer learning. Provides the cognitive modeling framework for personalized learning adaptation.

Key Data Structures and Relationships

Core Learning Components

adaptive: Adaptive scheduling with Expected Information Gain optimization
bayesian: Bayesian cognitive modeling and uncertainty quantification
learner: Core learner model with node embeddings and operation proficiencies
hierarchical_bayes: Hierarchical Bayesian models for population learning
strategy_mixture: Multiple strategy learning and mixture model fitting
transfer_learning: Cross-domain knowledge transfer and generalization
macro_learning: Meta-learning and strategy discovery algorithms

Cognitive Architecture

LearnerModel → NodeEmbeddings + OperationProficiencies + MemoryStrengths
AdaptiveScheduler → BayesianModel + TaskSelection + ModelUpdating

Main Data Flows and Transformations

Learning Pipeline

Task Selection: EIG-based optimization for maximum learning efficiency
Response Processing: Bayesian updating of cognitive model parameters
Strategy Adaptation: Dynamic strategy mixture based on performance patterns
Transfer Integration: Cross-domain knowledge application and generalization

Model Evolution

Parameter Updating: Continuous refinement of cognitive model parameters
Uncertainty Reduction: Information gain-based learning optimization
Strategy Discovery: Automatic identification of effective learning strategies
Knowledge Transfer: Application of learned patterns to new domains

Core Algorithms and Business Logic Abstractions

Bayesian Inference: Posterior updating with evidence integration
Information Theory: Expected Information Gain calculation for task selection
Mixture Modeling: Multiple strategy identification and weighting
Hierarchical Modeling: Individual differences within population structure

Adaptive Learning Module Abstract

High-level Purpose and Responsibility

The adaptive learning module implements intelligent task selection and difficulty adaptation using multi-armed bandit algorithms and reinforcement learning principles. It dynamically adjusts learning experiences based on real-time performance feedback to optimize learning efficiency and maintain appropriate challenge levels for individual learners.

Key Data Structures and Relationships

AdaptiveLearner: Core adaptive learning system with bandit algorithm integration
ThompsonSampler: Bayesian bandit algorithm for exploration-exploitation balance
UCBLearner: Upper Confidence Bound algorithm for optimistic task selection
DifficultyController: Manages task difficulty progression based on performance patterns
ExplorationStrategy: Balances between known effective tasks and exploration of new challenges
PerformancePredictor: Forecasts learning outcomes for different task selections

Main Data Flows and Transformations

Task Selection: Learner state → Bandit algorithm → Optimal task recommendation
Performance Integration: Task outcomes → Reward signals → Algorithm parameter updates
Difficulty Adaptation: Performance trends → Dynamic difficulty adjustment recommendations
Exploration Management: Uncertainty estimates → Controlled exploration of learning space
Strategy Optimization: Long-term outcomes → Meta-learning of selection strategies

External Dependencies and Interfaces

Learning Module: Integration with core learner state and proficiency tracking
Tasks Module: Task generation with adaptive difficulty and content selection
Statistics Module: Bayesian inference for Thompson sampling and confidence bounds
Experiments Module: A/B testing framework for strategy comparison and validation

State Management Patterns

Bandit State Updates: Incremental learning of task-outcome relationships
Exploration-Exploitation Balance: Dynamic adjustment of exploration rates based on learning progress
Difficulty Trajectory Tracking: Maintains optimal challenge progression over time
Strategy Memory: Retains learned preferences and successful adaptation patterns

Core Algorithms or Business Logic Abstractions

Thompson Sampling: Bayesian bandit algorithm with posterior sampling for task selection
Upper Confidence Bounds: Optimistic selection with confidence interval-based exploration
Contextual Bandits: Task selection incorporating learner state and historical performance
Difficulty Calibration: Real-time adjustment of task parameters to maintain optimal challenge
Regret Minimization: Algorithms designed to minimize learning inefficiency over time
Multi-Objective Optimization: Balancing multiple learning objectives (accuracy, speed, retention)

Bayesian Learning Module Abstract

High-level Purpose and Responsibility

The Bayesian learning module implements probabilistic learning models that maintain explicit uncertainty estimates about learner proficiency and skill acquisition. It provides principled belief updates through Bayesian inference, enabling robust learning assessment and prediction under uncertainty with proper handling of limited data scenarios.

Key Data Structures and Relationships

BayesianLearner: Probabilistic learner model with belief distributions over skill parameters
BeliefState: Current probability distributions representing learner skill estimates
PriorDistribution: Initial beliefs about learner capabilities before observing performance
PosteriorUpdate: Bayesian belief revision mechanism incorporating new evidence
CredibleInterval: Uncertainty quantification for proficiency estimates
PredictiveDistribution: Forward-looking probability distributions for performance prediction

Main Data Flows and Transformations

Prior Specification: Domain knowledge → Initial belief distributions over learner parameters
Likelihood Computation: Task performance → Probability of observed data given skill levels
Posterior Update: Prior beliefs + New evidence → Updated skill probability distributions
Prediction Generation: Current beliefs → Probabilistic forecasts of future performance
Uncertainty Propagation: Belief updates → Confidence intervals and decision-making support

External Dependencies and Interfaces

Statistics Module: Probability distributions, Bayesian inference algorithms, and numerical integration
Learning Module: Integration with core learner proficiency tracking and state management
Tasks Module: Likelihood models connecting task difficulty to performance probabilities
Experiments Module: Bayesian experimental design and optimal data collection strategies

State Management Patterns

Belief Distribution Maintenance: Continuous updating of probability distributions over skill parameters
Conjugate Prior Updates: Efficient analytical updates when using conjugate prior-likelihood pairs
Non-Conjugate Inference: Numerical methods (MCMC, variational inference) for complex belief updates
Hierarchical Belief Structure: Multi-level beliefs incorporating individual and population-level parameters

Core Algorithms or Business Logic Abstractions

Bayesian Parameter Estimation: Maximum a posteriori (MAP) and posterior mean estimation of skill levels
Predictive Modeling: Posterior predictive distributions for performance forecasting
Model Selection: Bayesian model comparison for choosing optimal learning representations
Sequential Belief Updates: Online Bayesian learning with streaming performance data
Uncertainty Quantification: Credible intervals, prediction intervals, and decision-theoretic uncertainty measures
Hyperparameter Learning: Hierarchical Bayes for learning prior parameters from data

Hierarchical Bayesian Learning Module Abstract

High-level Purpose and Responsibility

The hierarchical Bayesian learning module implements multi-level probabilistic models that capture both individual learner differences and population-level learning patterns. It enables sharing of statistical strength across learners while maintaining individual adaptation, supporting robust learning assessment for both well-studied and new learners through principled statistical borrowing.

Key Data Structures and Relationships

HierarchicalModel: Multi-level Bayesian model with population and individual parameter layers
PopulationParameters: Group-level hyperpriors representing shared learning characteristics
IndividualParameters: Learner-specific parameters drawn from population distributions
HyperpriorSpecification: Prior beliefs about population-level learning parameters
SharedLearningDynamics: Common learning patterns shared across the learner population
IndividualVariability: Person-specific deviations from population learning trends

Main Data Flows and Transformations

Hierarchical Inference: Individual observations → Joint estimation of population and individual parameters
Shrinkage Effects: Individual estimates → Regularization toward population means based on data quantity
Population Learning: Aggregate learner data → Refined understanding of general learning patterns
Individual Adaptation: Person-specific performance → Customized learning parameter estimation
Predictive Synthesis: Population + Individual knowledge → Enhanced performance predictions for new learners

External Dependencies and Interfaces

Statistics Module: MCMC sampling, variational inference, and hierarchical model estimation techniques
Bayesian Module: Individual-level Bayesian learning components and belief update mechanisms
Learning Module: Integration with core learner state management and proficiency tracking
Experiments Module: Population-level experimental design and multi-learner study coordination

State Management Patterns

Multi-Level Parameter Storage: Simultaneous maintenance of population and individual parameter estimates
Conjugate Hierarchical Updates: Efficient analytical updates for conjugate hierarchical models
Non-Conjugate Sampling: MCMC and variational methods for complex hierarchical inference
Dynamic Population Updates: Incremental population parameter learning as new learners join

Core Algorithms or Business Logic Abstractions

Empirical Bayes Estimation: Data-driven learning of population hyperparameters
Full Bayesian Hierarchy: Complete uncertainty propagation through all model levels
Shrinkage Estimation: Optimal combination of individual and population information
Random Effects Modeling: Individual learner effects drawn from population distributions
Meta-Learning: Population-level learning about learning processes and individual differences
Predictive Modeling for New Learners: Leveraging population knowledge for cold-start prediction

Learner Core Module Abstract

High-level Purpose and Responsibility

The learner core module provides the fundamental framework for modeling individual learner states and tracking learning progress across different cognitive operations. It serves as the foundational abstraction for all learning algorithms, maintaining proficiency estimates, operation-specific performance tracking, and chunk boundary awareness for sequence learning tasks.

Key Data Structures and Relationships

Learner: Central learner model containing proficiency tracking and learning history
LearnerState: Current learning state with performance metrics and confidence estimates
OperationType: Enumeration of cognitive operations (successor, predecessor, segment recitation, etc.)
ProficiencyLevel: Quantified skill levels with confidence intervals and mastery thresholds
ChunkBoundary: Cognitive boundaries in sequence learning with strength and position information
LearningHistory: Temporal record of performance changes and skill acquisition patterns

Main Data Flows and Transformations

Performance Update: Task responses → Proficiency level adjustments using learning algorithms
Operation Classification: Task types → Specific operation tracking and skill specialization
Confidence Estimation: Response patterns → Uncertainty quantification for proficiency estimates
Boundary Detection: Sequence performance → Identification of chunk boundaries and cognitive organization
Skill Transfer: Cross-operation learning → Transfer effects and skill generalization patterns

External Dependencies and Interfaces

Statistics Module: Statistical inference for proficiency estimation and confidence intervals
Tasks Module: Task classification and difficulty assessment for learning calibration
Experiments Module: Integration with experimental designs and outcome measurement
Data Module: Persistent storage of learning states and historical performance data

State Management Patterns

Incremental Learning Updates: Continuous refinement of proficiency estimates with new observations
Operation-Specific Tracking: Separate skill progression for different cognitive operations
Confidence-Weighted Updates: Learning rate adaptation based on estimate uncertainty
Boundary-Aware Learning: Special handling for chunk boundaries and hierarchical skill organization

Core Algorithms or Business Logic Abstractions

Proficiency Estimation: Bayesian inference for skill level with uncertainty quantification
Learning Rate Adaptation: Dynamic adjustment of learning parameters based on performance consistency
Skill Transfer Modeling: Cross-operation knowledge transfer and generalization patterns
Mastery Threshold Detection: Automatic identification of skill acquisition milestones
Chunk Boundary Learning: Detection and utilization of cognitive organization in sequence learning
Performance Prediction: Forward modeling of expected performance based on current learner state

Macro Learning Module Abstract

High-level Purpose and Responsibility

The macro learning module implements automated discovery and learning of higher-order patterns and chunks in sequential learning tasks. It identifies recurring subsequences, builds hierarchical representations of learned material, and enables efficient learning and recall through pattern recognition and chunking strategies.

Key Data Structures and Relationships

MacroDiscovery: Pattern detection system for identifying recurring subsequences in learning tasks
ChunkHierarchy: Multi-level representation of learned patterns from individual items to complex macros
PatternLibrary: Repository of discovered patterns with frequency and utility statistics
MacroApplication: System for applying learned patterns to accelerate new learning
SequenceDecomposition: Breaking down complex sequences into learned chunks and novel elements
PatternGeneralization: Extension of specific patterns to broader contexts and variations

Main Data Flows and Transformations

Pattern Detection: Learning sequences → Statistical analysis → Discovery of recurring subsequences
Chunk Formation: Detected patterns → Hierarchical organization → Multi-level pattern library
Pattern Application: New sequences → Pattern matching → Acceleration of learning through chunking
Hierarchical Compression: Low-level patterns → Composition → Higher-order macro structures
Transfer Learning: Learned patterns → Application to novel domains → Cross-context skill transfer

External Dependencies and Interfaces

Learning Module: Integration with core learner state for pattern-based skill development
Tasks Module: Sequence analysis and generation leveraging discovered patterns
Statistics Module: Statistical significance testing for pattern discovery and frequency analysis
Protocol Module: Versioning and reproducibility of discovered pattern libraries

State Management Patterns

Incremental Pattern Discovery: Continuous updating of pattern library with new learning experiences
Pattern Strength Tracking: Maintains usage frequency and success rates for different patterns
Hierarchical Pattern Organization: Multi-level storage and retrieval of patterns at different abstraction levels
Context-Dependent Activation: Dynamic pattern selection based on current learning context

Core Algorithms or Business Logic Abstractions

Suffix Tree Construction: Efficient data structures for pattern discovery in sequential data
Statistical Pattern Mining: Frequency-based and significance-based pattern identification algorithms
Hierarchical Clustering: Organization of patterns into meaningful hierarchical structures
Pattern Compression: Identification of optimal chunking strategies for memory efficiency
Context-Sensitive Pattern Application: Dynamic selection of patterns based on current learning context
Meta-Pattern Discovery: Learning about patterns of patterns and higher-order regularities

Strategy Mixture Learning Module Abstract

High-level Purpose and Responsibility

The strategy mixture learning module implements ensemble learning approaches that combine multiple learning strategies and algorithms to achieve robust and adaptive learning performance. It dynamically weights different learning approaches based on their contextual effectiveness, enabling the system to leverage the strengths of diverse learning paradigms.

Key Data Structures and Relationships

StrategyMixture: Ensemble controller managing multiple learning strategies with dynamic weighting
LearningStrategy: Abstract interface for individual learning algorithms (Bayesian, adaptive, etc.)
StrategyWeights: Dynamic importance weights for different strategies based on performance context
ContextualSelector: Context-aware strategy selection and weighting mechanism
EnsemblePrediction: Combined predictions from multiple strategies with uncertainty quantification
StrategyPerformance: Performance tracking for individual strategies across different contexts

Main Data Flows and Transformations

Strategy Coordination: Learning context → Strategy activation and weight assignment
Ensemble Prediction: Multiple strategy outputs → Weighted combination → Final learning recommendations
Weight Adaptation: Strategy performance feedback → Dynamic reweighting of ensemble components
Context Recognition: Learning situation → Optimal strategy mixture selection
Meta-Learning: Long-term strategy performance → Learning about strategy effectiveness patterns

External Dependencies and Interfaces

Learning Module: Integration with all individual learning strategy implementations
Adaptive Module: Thompson sampling and bandit algorithms as mixture components
Bayesian Module: Probabilistic learning strategies for ensemble inclusion
Statistics Module: Weight optimization and ensemble combination algorithms

State Management Patterns

Multi-Strategy State Synchronization: Coordinated state updates across all component learning strategies
Dynamic Weight Evolution: Continuous adaptation of strategy importance based on performance
Context-Dependent Configuration: Different mixture weights for different learning contexts
Meta-Learning State: Higher-order learning about strategy combination effectiveness

Core Algorithms or Business Logic Abstractions

Weighted Ensemble Methods: Optimal combination of predictions from multiple learning strategies
Online Weight Learning: Adaptive algorithms for learning strategy mixture weights
Context-Aware Strategy Selection: Dynamic activation of strategies based on learning situation
Multi-Objective Optimization: Balancing multiple learning goals across strategy components
Ensemble Uncertainty Quantification: Combining uncertainty estimates from multiple strategies
Meta-Strategy Learning: Learning about when and how to combine different learning approaches

Transfer Learning Module Abstract

High-level Purpose and Responsibility

The transfer learning module enables knowledge and skill transfer across different learning domains and task types. It identifies transferable learning components, maps skills between domains, and accelerates learning in new contexts by leveraging previously acquired knowledge and learning strategies.

Key Data Structures and Relationships

TransferLearner: Core system for managing cross-domain knowledge transfer
DomainMapping: Structural relationships between source and target learning domains
SkillTransferMatrix: Quantified transfer relationships between different cognitive skills
TransferableKnowledge: Abstract knowledge representations that generalize across domains
ContextBridge: Mechanisms for mapping concepts and skills between different learning contexts
TransferEffectiveness: Measurement and tracking of knowledge transfer success rates

Main Data Flows and Transformations

Domain Analysis: Source and target domains → Structural similarity assessment → Transfer feasibility
Knowledge Extraction: Source domain expertise → Abstract skill representations → Transferable components
Cross-Domain Mapping: Source skills → Domain mapping → Target domain skill initialization
Transfer Application: Transferable knowledge → Target domain adaptation → Accelerated learning
Transfer Validation: Learning outcomes → Transfer effectiveness measurement → Strategy refinement

External Dependencies and Interfaces

Learning Module: Integration with core learner state and proficiency tracking across domains
Tasks Module: Cross-domain task generation and skill assessment capabilities
Statistics Module: Transfer effectiveness measurement and statistical validation
Protocol Module: Reproducible transfer learning protocols and domain specifications

State Management Patterns

Multi-Domain State Management: Simultaneous tracking of learning progress across multiple domains
Transfer History Tracking: Recording successful and failed transfer attempts for meta-learning
Adaptive Transfer Strategy: Dynamic adjustment of transfer approaches based on effectiveness
Domain-Specific State Isolation: Separate state management for domain-specific vs. transferable knowledge

Core Algorithms or Business Logic Abstractions

Similarity Assessment: Algorithms for measuring structural and functional similarity between domains
Abstract Skill Extraction: Methods for identifying domain-independent learning components
Transfer Mapping Algorithms: Systematic approaches for mapping skills and knowledge across domains
Transfer Effectiveness Prediction: Models for predicting the success of proposed knowledge transfers
Negative Transfer Detection: Identification and mitigation of harmful cross-domain interference
Meta-Transfer Learning: Learning about transfer learning itself to improve future transfer attempts

Tasks Module - Abstract Documentation

Purpose and Responsibility

Implements comprehensive task generation and management system supporting multiple task types across different topology structures. Provides adaptive difficulty scaling, boundary analysis, and extended cognitive tasks for research applications.

Key Data Structures and Relationships

Task Components

core: Core task types and generation framework
boundaries: Chunk boundary analysis and crossing detection
extended: Advanced task types and dynamic topology manipulation
music: Musical sequence learning and pattern recognition tasks
navigation: Spatial and abstract navigation task generation

Task Hierarchy

TaskGenerator → TaskType → Task → Response → Performance Analysis
TaskSession → TaskHistory → SessionStatistics → LearningAnalysis

Main Data Flows and Transformations

Task Generation Pipeline

Type Selection: Adaptive or random task type selection based on learning model
Parameter Generation: Difficulty-appropriate parameter selection for task instance
Validation: Task solvability and appropriateness verification
Presentation: User-friendly task formatting with clear instructions

Response Processing

Answer Validation: Correctness checking with partial credit support
Timing Analysis: Response time measurement and reaction time modeling
Learning Integration: Performance feedback to adaptive learning algorithms
Quality Assessment: Response validity and engagement measurement

Core Algorithms and Business Logic Abstractions

Difficulty Calibration: Adaptive difficulty adjustment based on performance
Boundary Detection: Chunk boundary identification and crossing analysis
Task Sequencing: Optimal task ordering for learning effectiveness
Transfer Task Generation: Cross-domain task creation for generalization testing

Task Generation Core Module Abstract

High-level Purpose and Responsibility

The task generation core module provides the fundamental framework for creating and managing learning tasks based on topological structures. It serves as the central task generation engine, creating diverse cognitive challenges including sequence learning, spatial navigation, and logical reasoning tasks while maintaining consistent difficulty calibration and performance tracking.

Key Data Structures and Relationships

Task: Core task representation with prompt, correct answer, options, and difficulty rating
TaskType: Enumeration of different cognitive task categories (sequence, navigation, reasoning)
TaskGenerator: Central factory for creating tasks based on topological structures and parameters
TaskResponse: Response tracking with accuracy, timing, and metadata for performance analysis
TaskSession: Session management for task sequences with history and statistical tracking
SessionStatistics: Comprehensive performance analytics across task types and sessions

Main Data Flows and Transformations

Task Generation: Topology + Task type specifications → Algorithmic task creation → Formatted learning challenges
Response Collection: User interactions → Task responses → Performance data with timing information
Difficulty Calibration: Task parameters → Dynamic difficulty assessment → Appropriately challenging tasks
Performance Tracking: Response history → Statistical analysis → Learner proficiency assessment
Session Management: Task sequences → Progress tracking → Comprehensive session analytics

External Dependencies and Interfaces

Topology Module: Structural foundations for task generation based on graph relationships
Learning Module: Integration with learner proficiency tracking and adaptive difficulty
Statistics Module: Performance analysis and statistical assessment of task outcomes
Experiments Module: Task provision for experimental studies and controlled comparisons

State Management Patterns

Immutable Task Definitions: Task specifications remain constant after generation
Session State Tracking: Maintains current task, timing, and historical performance data
Random Generation State: Controlled randomization for reproducible task sequences
Performance History Management: Accumulates response data for longitudinal analysis

Core Algorithms or Business Logic Abstractions

Topological Task Generation: Systematic creation of tasks based on graph structure and relationships
Difficulty Assessment: Dynamic calculation of task difficulty based on cognitive load and complexity
Random Task Selection: Balanced selection algorithms ensuring diverse task coverage
Performance Analysis: Real-time computation of accuracy rates, response times, and learning trends
Adaptive Task Sequencing: Intelligent ordering of tasks based on learner progress and proficiency
Multi-Modal Task Support: Generation framework supporting various task types and interaction modalities

Extended Task Generation Module Abstract

High-level Purpose and Responsibility

The extended task generation module implements advanced cognitive challenges that go beyond basic sequence learning to include complex reasoning, dynamic topology manipulation, and transfer learning tasks. It provides sophisticated learning challenges for higher-order cognitive skills including analogical reasoning, pattern recognition, and adaptive problem-solving.

Key Data Structures and Relationships

ExtendedTaskGenerator: Advanced task creation system for complex cognitive challenges
ReasoningTask: Tasks requiring logical inference and multi-step reasoning processes
TopologyManipulation: Dynamic modification of learning structures during task execution
TransferTask: Cross-domain learning challenges requiring skill generalization
AnalogicalMapping: Structural correspondence identification between different domains
ComplexityMetric: Multi-dimensional assessment of cognitive challenge complexity

Main Data Flows and Transformations

Reasoning Task Creation: Logic patterns → Multi-step inference challenges → Complex reasoning tasks
Dynamic Topology Generation: Base structures → Real-time modifications → Adaptive learning environments
Transfer Challenge Design: Source domain skills → Cross-domain mapping → Transfer learning tasks
Analogical Task Construction: Structural patterns → Analogical relationships → Mapping challenges
Complexity Calibration: Multiple complexity dimensions → Integrated difficulty assessment → Appropriately challenging tasks

External Dependencies and Interfaces

Learning Module: Integration with advanced learning algorithms and transfer learning capabilities
Tasks Module: Extension of core task generation with sophisticated cognitive challenges
Topology Module: Dynamic topology manipulation and structural reasoning support
Statistics Module: Complex performance analysis for multi-dimensional cognitive assessments

State Management Patterns

Dynamic Topology State: Real-time management of changing topological structures during tasks
Transfer Context Tracking: Maintains context information for cross-domain learning challenges
Reasoning Chain Management: Tracks multi-step reasoning processes and intermediate states
Complexity State Evolution: Dynamic adjustment of task complexity based on learner progress

Core Algorithms or Business Logic Abstractions

Multi-Step Reasoning Generation: Creation of tasks requiring sequential logical inferences
Dynamic Structure Modification: Real-time topology changes for adaptive learning environments
Analogical Mapping Algorithms: Systematic identification of structural correspondences between domains
Transfer Challenge Design: Creation of tasks requiring application of skills in novel contexts
Complexity Synthesis: Integration of multiple complexity dimensions into unified difficulty metrics
Meta-Cognitive Task Creation: Tasks that require thinking about thinking and learning strategies

Music Theory Task Generation Module Abstract

High-level Purpose and Responsibility

The music theory task generation module creates specialized learning tasks based on musical structures and relationships. It implements comprehensive music theory education through topological representations of scales, chord progressions, and harmonic relationships, supporting both theoretical knowledge and practical musical skill development.

Key Data Structures and Relationships

MusicTheory: Comprehensive musical knowledge representation with scales, chords, and intervals
MusicStructure: Enumeration of musical forms (chromatic scale, circle of fifths, chord progressions)
NoteAttributes: Complete note information including pitch class, frequency, enharmonics, and synesthetic colors
Interval: Musical interval representation with semitone distance, quality, and consonance ratings
MusicTaskGenerator: Specialized task creation system for music theory learning
ChordProgression: Sequential harmonic structures with voice leading and substitution patterns

Main Data Flows and Transformations

Musical Structure Creation: Music theory → Topological representation → Task-ready musical graphs
Theory Task Generation: Musical concepts → Educational tasks → Music theory learning challenges
Interval Analysis: Note relationships → Harmonic analysis → Interval recognition tasks
Progression Training: Chord sequences → Harmonic progression → Musical structure learning
Cross-Modal Integration: Musical concepts → Synesthetic associations → Multi-sensory learning tasks

External Dependencies and Interfaces

Topology Module: Graph representation of musical structures and harmonic relationships
Tasks Module: Integration with core task generation framework for educational delivery
Learning Module: Adaptation to individual musical learning progress and skill development
Statistics Module: Analysis of musical learning patterns and harmonic comprehension metrics

State Management Patterns

Musical Knowledge Base: Persistent storage of comprehensive music theory information
Harmonic Relationship Mapping: Maintains complex interval and chord relationship networks
Scale and Mode Management: Organized storage of different musical scales and modal systems
Progression State Tracking: Monitors learner progress through different musical complexity levels

Core Algorithms or Business Logic Abstractions

Automatic Scale Generation: Algorithmic creation of scales from interval patterns and root notes
Harmonic Analysis Algorithms: Systematic analysis of chord progressions and voice leading
Interval Calculation: Mathematical computation of musical intervals with enharmonic handling
Circle of Fifths Navigation: Topological representation and navigation of key relationships
Chord Progression Generation: Rule-based creation of harmonically coherent chord sequences
Synesthetic Color Mapping: Integration of color associations for enhanced musical memory and learning

High-level Purpose and Responsibility

The constrained navigation task generation module creates sophisticated spatial reasoning and pathfinding challenges with complex constraint systems. It implements multi-hop reasoning tasks, property-based navigation constraints, and advanced pathfinding algorithms to develop spatial cognitive skills and logical reasoning abilities.

Key Data Structures and Relationships

ConstrainedNavigator: Advanced pathfinding system with complex constraint satisfaction
NavigationConstraint: Comprehensive constraint types including avoidance, requirements, and property filters
ConditionalNavigationTask: Navigation challenges with multiple simultaneous constraints
MultiHopNavigator: System for complex multi-step reasoning and spatial relationship analysis
PropertyFilter: Sophisticated filtering system based on node properties and spatial characteristics
ConstraintSatisfaction: Algorithms for finding paths that satisfy multiple simultaneous constraints

Main Data Flows and Transformations

Constraint Specification: Navigation requirements → Formal constraint representation → Pathfinding constraints
Path Search: Start/goal positions + Constraints → A* with constraint checking → Valid navigation paths
Multi-Hop Analysis: Spatial relationships → Complex reasoning challenges → Multi-step navigation tasks
Property-Based Filtering: Node characteristics → Property-based constraints → Filtered navigation options
Reasoning Task Generation: Spatial structures → Complex logical challenges → Advanced cognitive tasks

External Dependencies and Interfaces

Topology Module: Spatial graph structures and pathfinding algorithms for navigation challenges
Tasks Module: Integration with core task generation for educational delivery and assessment
Learning Module: Adaptive difficulty based on spatial reasoning skill development
Statistics Module: Analysis of spatial reasoning performance and constraint satisfaction abilities

State Management Patterns

Constraint State Management: Dynamic tracking of active constraints and satisfaction status
Path Search State: A* algorithm state with constraint-aware search space exploration
Property Database Management: Maintains comprehensive node property information for filtering
Multi-Hop Reasoning State: Tracks complex reasoning chains and intermediate conclusions

Core Algorithms or Business Logic Abstractions

Constraint-Aware A Search*: Modified A* pathfinding with real-time constraint satisfaction checking
Multi-Constraint Optimization: Algorithms for satisfying multiple simultaneous navigation constraints
Property-Based Filtering: Sophisticated node filtering based on complex property specifications
Equidistant Point Discovery: Geometric algorithms for finding nodes with equal distances from multiple sources
Complex Reasoning Generation: Creation of tasks requiring multi-step logical inferences about spatial relationships
Constraint Conflict Resolution: Systematic approaches to handling conflicting or impossible constraint combinations

Boundary-Aware Task Generation Module Abstract

High-level Purpose and Responsibility

The boundary-aware task generation module creates specialized learning tasks that target cognitive chunk boundaries and hierarchical organization in sequence learning. It implements boundary detection, crossing challenges, and hierarchical chunking tasks to study and improve learners' ability to navigate mental boundaries and organizational structures.

Key Data Structures and Relationships

BoundaryTrainer: Central system for generating boundary-specific learning challenges
ChunkBoundary: Representation of cognitive boundaries with position and strength metrics
BoundaryType: Classification of different boundary types (chunk, category, hierarchical)
HierarchicalChunker: Multi-level chunking system for complex boundary hierarchies
BoundaryBridgingTask: Tasks specifically designed to cross and integrate across boundaries
ChunkLevel: Hierarchical organization of chunks at different abstraction levels

Main Data Flows and Transformations

Boundary Detection: Sequence structure → Statistical boundary identification → Cognitive boundary mapping
Task Specialization: Boundary information → Targeted task generation → Boundary-crossing challenges
Hierarchical Organization: Flat sequences → Multi-level chunking → Hierarchical task structures
Integration Training: Boundary-crossing tasks → Seamless integration practice → Improved fluency
Difficulty Progression: Boundary strength → Graduated challenge levels → Adaptive boundary training

External Dependencies and Interfaces

Learning Module: Integration with chunk boundary awareness in learner proficiency tracking
Tasks Module: Extension of core task generation with boundary-specific algorithms
Statistics Module: Boundary strength quantification and crossing difficulty analysis
Topology Module: Structural analysis for boundary identification and strength assessment

State Management Patterns

Boundary Map Maintenance: Persistent tracking of identified boundaries and their characteristics
Hierarchical State Management: Multi-level organization of boundaries at different abstraction levels
Training Progress Tracking: Monitors learner improvement in boundary-crossing abilities
Adaptive Boundary Selection: Dynamic selection of appropriate boundaries for training

Core Algorithms or Business Logic Abstractions

Statistical Boundary Detection: Algorithms for identifying cognitive boundaries using performance patterns
Hierarchical Decomposition: Systematic organization of sequences into multi-level chunk hierarchies
Boundary Crossing Generation: Specialized task creation targeting specific boundary transitions
Integration Challenge Creation: Tasks requiring seamless navigation across multiple boundaries
Difficulty Calibration: Dynamic adjustment of boundary-crossing challenge based on boundary strength
Oscillation Training: Rapid back-and-forth boundary crossing for fluency development

Experiments Module - Abstract Documentation

Purpose and Responsibility

Provides comprehensive experimental framework for conducting controlled learning research studies. Manages experimental design, participant assignment, condition randomization, data collection, and statistical analysis for rigorous research applications.

Key Data Structures and Relationships

Core Framework Components

core: Main experimental framework with experiment management and execution
ab_testing: A/B testing functionality for comparing learning conditions
design: Experimental design patterns and statistical power analysis
multi_session: Longitudinal study management across multiple sessions

Experimental Hierarchy

ExperimentFramework → Experiments → Conditions → Participants → Sessions → Trials

Main Data Flows and Transformations

Experiment Lifecycle

Design Phase: Experimental design, power analysis, condition specification
Setup Phase: Participant recruitment, randomization, condition assignment
Execution Phase: Data collection, session management, real-time monitoring
Analysis Phase: Statistical analysis, effect size calculation, report generation

Data Collection Pipeline

Session Management: Multi-session studies with participant tracking
Condition Control: Randomized assignment and counterbalancing
Quality Assurance: Real-time data validation and integrity checks
Export Integration: Research-ready data formats for analysis software

External Dependencies and Interfaces

Statistical Framework: Integration with statistics module for analysis
Learning System: Core learning models and adaptive algorithms
Task Generation: Experimental task creation and difficulty calibration
Data Export: Research-standard data formats and analysis scripts

Core Algorithms and Business Logic Abstractions

Randomization: Condition assignment with counterbalancing and blocking
Power Analysis: Sample size calculation and effect size estimation
Multi-level Analysis: Hierarchical modeling for nested experimental designs
Longitudinal Tracking: Participant progress across multiple sessions

Experiment Core Module Abstract

High-level Purpose and Responsibility

The experiments core module provides the foundational infrastructure for managing experimental workflows in the learning system. It serves as the central orchestrator for experiment lifecycle management, data collection, and result computation, supporting various experimental paradigms including A/B testing, multi-session studies, and adaptive learning experiments.

Key Data Structures and Relationships

Experiment: Central struct containing experiment metadata, configuration, state, and collected data
ExperimentConfig: Configuration parameters including participant limits, task parameters, and randomization settings
ExperimentResult: Computed outcomes including statistical measures, performance metrics, and effect sizes
ExperimentState: Runtime state management (NotStarted, Running, Paused, Completed)
TaskResponse: Individual response records linking tasks to participant behavior
ParticipantData: Aggregated participant information and performance history

Main Data Flows and Transformations

Experiment Creation: Configuration → Experiment instantiation with participant management
Task Generation: Experiment parameters → Task generation through TaskGenerator integration
Response Collection: Participant inputs → TaskResponse records with timing and accuracy data
Result Computation: Collected responses → Statistical analysis and performance metrics
State Management: External events → State transitions with validation and persistence

External Dependencies and Interfaces

Learning Module: Integration with adaptive learning algorithms and learner state management
Tasks Module: Task generation and validation through TaskGenerator and TaskSession
Statistics Module: Statistical analysis, hypothesis testing, and effect size calculations
Data Module: Persistence layer for experiment data and participant tracking
Protocol Module: Version control and reproducibility through seed management

State Management Patterns

Finite State Machine: Experiment lifecycle managed through explicit state transitions
Event-driven Updates: Participant actions trigger state changes and data collection
Immutable Results: Computed results are immutable once experiment is completed
Concurrent Participant Management: Thread-safe participant data collection and aggregation

Core Algorithms or Business Logic Abstractions

Adaptive Sampling: Dynamic participant assignment based on experiment design and statistical power
Real-time Analysis: Incremental computation of performance metrics during experiment execution
Randomization Control: Deterministic randomization using configurable seeds for reproducibility
Effect Size Calculation: Standardized effect size computation (Cohen's d, eta-squared) for result interpretation
Completion Criteria: Automated experiment termination based on statistical significance or sample size targets

Experiment Design Module Abstract

High-level Purpose and Responsibility

The experiment design module implements various experimental design paradigms for learning research. It provides structured frameworks for organizing participants, treatments, and conditions across different experimental methodologies, ensuring proper randomization, control, and statistical validity in learning experiments.

Key Data Structures and Relationships

ExperimentDesign: Trait defining common behavior for all experimental designs
WithinSubjectDesign: All participants experience all conditions (repeated measures)
BetweenSubjectDesign: Participants are assigned to single conditions (independent groups)
MixedDesign: Combines within and between-subject factors in factorial arrangements
FactorialDesign: Full factorial crossing of multiple independent variables
TreatmentAssignment: Maps participants to specific experimental conditions
ConditionDefinition: Specifies parameters and constraints for each experimental condition

Main Data Flows and Transformations

Participant Assignment: Participant IDs → Condition assignments through randomization algorithms
Condition Ordering: Design specifications → Counterbalanced condition sequences
Treatment Application: Assigned conditions → Specific task parameters and learning contexts
Balance Validation: Assignment outcomes → Statistical balance checks across conditions
Design Compliance: Runtime monitoring → Adherence to design constraints and protocols

External Dependencies and Interfaces

Random Number Generation: Deterministic randomization using controlled seeds
Statistics Module: Balance testing and power analysis for design validation
Learning Module: Integration with adaptive learning parameters per condition
Protocol Module: Version tracking for design specifications and changes

State Management Patterns

Immutable Design Specifications: Design parameters fixed at instantiation
Dynamic Assignment State: Participant assignments updated during enrollment
Condition Tracking: Real-time monitoring of participant progress across conditions
Balance Maintenance: Automatic rebalancing when participants drop out or are excluded

Core Algorithms or Business Logic Abstractions

Randomization Algorithms: Block randomization, stratified randomization, and minimization procedures
Counterbalancing Logic: Systematic ordering of conditions to control for sequence effects
Power Analysis Integration: Sample size calculations based on expected effect sizes and design complexity
Factorial Expansion: Automatic generation of all condition combinations in factorial designs
Assignment Validation: Ensures proper distribution of participants across conditions
Crossover Sequence Generation: Latin square and other systematic ordering schemes for within-subject designs

A/B Testing Module Abstract

High-level Purpose and Responsibility

The A/B testing module provides specialized infrastructure for conducting controlled experiments comparing two or more variants of learning interventions. It implements both classical and Bayesian A/B testing frameworks with real-time statistical monitoring, early stopping criteria, and effect size estimation for learning optimization studies.

Key Data Structures and Relationships

ABTest: Core A/B testing framework with variant management and statistical tracking
BayesianABTest: Bayesian implementation with posterior belief updates and credible intervals
TestVariant: Individual treatment conditions with performance tracking and sample management
ConversionMetric: Outcome measures including accuracy, response time, and learning rate metrics
StatisticalResult: Hypothesis test outcomes with p-values, confidence intervals, and effect sizes
BayesianResult: Posterior distributions, credible intervals, and probability of superiority

Main Data Flows and Transformations

Participant Assignment: Incoming participants → Random allocation to test variants
Performance Tracking: Task responses → Aggregated conversion metrics per variant
Statistical Testing: Accumulated data → Hypothesis tests and significance calculations
Bayesian Updates: New observations → Posterior belief updates using conjugate priors
Decision Making: Statistical results → Recommendations for variant selection or continued testing

External Dependencies and Interfaces

Statistics Module: Hypothesis testing, effect size calculations, and confidence interval estimation
Learning Module: Integration with learning metrics and performance indicators
Tasks Module: Task generation and response collection for each test variant
Data Module: Persistent storage of test results and participant assignments

State Management Patterns

Immutable Test Configuration: Test parameters fixed after initialization
Dynamic Performance Tracking: Real-time accumulation of conversion metrics
Statistical State Updates: Incremental computation of test statistics as data arrives
Early Stopping Monitoring: Continuous evaluation of stopping criteria and statistical power

Core Algorithms or Business Logic Abstractions

Sequential Testing: Continuous monitoring with alpha spending functions and early stopping rules
Bayesian Belief Updates: Conjugate prior-posterior calculations for rapid inference
Effect Size Estimation: Cohen's d, odds ratios, and other standardized effect measures
Multiple Comparisons Correction: Bonferroni, FDR, and other corrections for multiple variant testing
Power Analysis: Real-time power calculations and sample size recommendations
Confidence Interval Construction: Bootstrap, asymptotic, and exact methods for interval estimation

Multi-Session Experiment Module Abstract

High-level Purpose and Responsibility

The multi-session experiment module manages longitudinal learning studies spanning multiple training sessions over extended time periods. It implements spaced repetition algorithms, session scheduling optimization, and cross-session performance tracking to study long-term learning dynamics and retention patterns.

Key Data Structures and Relationships

MultiSessionExperiment: Orchestrates multiple learning sessions with scheduling and progress tracking
SessionPlan: Defines session sequence, timing intervals, and content progression
CrossSessionTracker: Monitors learning transfer and retention across session boundaries
SpacedRepetitionScheduler: Implements forgetting curve-based scheduling algorithms
RetentionAnalysis: Analyzes memory consolidation and forgetting patterns
ProgressionMetrics: Tracks learning velocity and mastery development over time

Main Data Flows and Transformations

Session Scheduling: Learning history → Optimal session timing using spaced repetition algorithms
Content Selection: Previous performance → Adaptive content difficulty and review priority
Cross-Session Analysis: Multi-session data → Retention curves and learning trajectory modeling
Performance Aggregation: Session-level metrics → Longitudinal performance trends and patterns
Intervention Timing: Real-time performance → Dynamic session adjustments and schedule optimization

External Dependencies and Interfaces

Learning Module: Adaptive learning algorithms and mastery criteria integration
Statistics Module: Time series analysis, retention modeling, and longitudinal statistics
Tasks Module: Task generation adapted for multi-session progression and review
Protocol Module: Session versioning and consistency across experimental timeframes

State Management Patterns

Session State Persistence: Maintains learning state across session boundaries
Schedule State Management: Tracks upcoming sessions and completion status
Performance History: Immutable record of cross-session learning progression
Adaptive State Updates: Dynamic schedule adjustments based on performance patterns

Core Algorithms or Business Logic Abstractions

Spaced Repetition Algorithms: SM-2, Anki-style scheduling, and forgetting curve optimization
Retention Modeling: Exponential decay models and memory strength estimation
Learning Curve Fitting: Power law and exponential models for skill acquisition trajectories
Optimal Scheduling: Reinforcement learning approaches to session timing optimization
Transfer Analysis: Cross-session skill transfer detection and measurement
Mastery Threshold Adaptation: Dynamic criteria adjustment based on long-term retention performance

Statistics Module - Abstract Documentation

Purpose and Responsibility

Provides comprehensive statistical analysis framework for research applications including mixed-effects modeling, power analysis, validation testing, and publication-ready statistical reporting.

Key Data Structures and Relationships

Statistical Components

core: Core statistical analysis with learning curves and strategy classification
mixed_effects: Mixed-effects modeling for hierarchical experimental data
power_analysis: Statistical power calculation and sample size estimation
prediction: Predictive modeling and cross-validation frameworks
validation: Statistical validation and assumption testing
math_validation: Mathematical correctness verification and numerical stability

Analysis Architecture

StatisticalAnalyzer → LearningCurves + StrategyAnalysis + ErrorPatterns + RTModeling
MixedEffectsModel → FixedEffects + RandomEffects + Covariance Structure
PowerAnalysis → EffectSize + SampleSize + Power + AlphaLevel

Main Data Flows and Transformations

Analysis Pipeline

Data Validation: Assumption testing and outlier detection
Descriptive Analysis: Summary statistics and exploratory data analysis
Inferential Testing: Hypothesis testing with multiple comparison correction
Model Fitting: Mixed-effects models with optimal structure selection

Research Integration

Publication Standards: APA-compliant statistical reporting
Effect Sizes: Cohen's d, eta-squared, and confidence interval calculation
Power Analysis: Pre-study planning and post-hoc sensitivity analysis
Reproducible Analysis: Complete statistical pipeline with version control

Core Algorithms and Business Logic Abstractions

Mixed-Effects Modeling: Hierarchical linear models with random effects
Learning Curve Analysis: Non-linear growth modeling and plateau detection
Strategy Classification: Model-based clustering and mixture modeling
Response Time Modeling: Ex-Gaussian distribution fitting and parameter estimation

Statistics Core Module Abstract

High-level Purpose and Responsibility

The statistics core module provides fundamental statistical functions and probability distributions for the experimental learning system. It serves as the mathematical foundation for all statistical inference, hypothesis testing, and probabilistic modeling throughout the platform, ensuring numerical accuracy and statistical validity.

Key Data Structures and Relationships

StatisticalFunction: Core statistical computations including descriptive statistics and probability functions
ProbabilityDistribution: Standard probability distributions with parameter estimation and sampling
HypothesisTest: Framework for statistical significance testing with multiple test procedures
ConfidenceInterval: Interval estimation procedures with various construction methods
EffectSize: Standardized effect size measures for practical significance assessment
StatisticalSummary: Comprehensive statistical summaries with diagnostic information

Main Data Flows and Transformations

Data Summarization: Raw observations → Descriptive statistics → Statistical summaries
Distribution Fitting: Empirical data → Parameter estimation → Fitted probability distributions
Hypothesis Testing: Data + Null hypothesis → Test statistics → P-values and decisions
Interval Estimation: Sample data → Confidence intervals → Uncertainty quantification
Effect Size Calculation: Group differences → Standardized measures → Practical significance assessment

External Dependencies and Interfaces

Learning Module: Statistical assessment of learning progress and proficiency estimation
Experiments Module: Hypothesis testing for experimental outcomes and treatment effects
Bayesian Module: Prior and posterior distribution support for Bayesian inference
Validation Module: Statistical validation and assumption checking for model appropriateness

State Management Patterns

Immutable Statistical Results: Statistical computations produce immutable result objects
Cached Distribution Parameters: Performance optimization for repeated distribution operations
Numerical Precision Management: Careful handling of floating-point precision in statistical calculations
Error Propagation: Systematic propagation of numerical errors through statistical procedures

Core Algorithms or Business Logic Abstractions

Descriptive Statistics: Mean, median, variance, skewness, and kurtosis with numerical stability
Probability Density Functions: Implementation of standard statistical distributions (normal, beta, gamma, etc.)
Statistical Tests: t-tests, chi-square tests, ANOVA, and non-parametric alternatives
Maximum Likelihood Estimation: Parameter estimation for probability distributions
Bootstrap Methods: Resampling procedures for robust statistical inference
Multiple Comparisons Correction: Bonferroni, FDR, and other corrections for simultaneous inference

Mathematical Validation Module Abstract

High-level Purpose and Responsibility

The mathematical validation module provides rigorous validation of mathematical computations and numerical algorithms used throughout the learning system. It implements property-based testing, numerical accuracy verification, and mathematical consistency checks to ensure computational reliability and prevent numerical errors that could compromise experimental results.

Key Data Structures and Relationships

MathematicalProperty: Formal mathematical properties that algorithms should satisfy
NumericalAccuracyTest: Procedures for validating numerical precision and stability
PropertyBasedTest: Generative testing framework for mathematical function validation
ConsistencyChecker: Verification of mathematical consistency across different computational paths
PrecisionAnalyzer: Analysis of numerical precision and error propagation in computations
AlgorithmValidator: Comprehensive validation suite for mathematical algorithms

Main Data Flows and Transformations

Property Specification: Mathematical requirements → Formal property definitions → Testable specifications
Generative Testing: Property specifications → Random test case generation → Automated validation
Precision Analysis: Numerical computations → Error analysis → Precision recommendations
Consistency Checking: Alternative implementations → Cross-validation → Consistency verification
Algorithm Validation: Mathematical functions → Comprehensive testing → Reliability certification

External Dependencies and Interfaces

Statistics Module: Validation of statistical computations and probability calculations
Learning Module: Mathematical validation of learning algorithms and optimization procedures
Experiments Module: Numerical validation of experimental computations and effect size calculations
Protocol Module: Validation of deterministic computations for reproducibility

State Management Patterns

Validation State Persistence: Maintains validation results for different mathematical components
Error Tolerance Management: Configurable precision requirements for different computational contexts
Test Case Caching: Performance optimization for repeated mathematical validation
Precision Tracking: Monitoring of numerical precision across different computational pathways

Core Algorithms or Business Logic Abstractions

Property-Based Testing: QuickCheck-style generative testing for mathematical properties
Numerical Differentiation: Validation of analytical derivatives using finite difference methods
Error Propagation Analysis: Systematic analysis of how numerical errors accumulate
Convergence Testing: Validation of iterative algorithms and optimization procedures
Invariant Checking: Verification that mathematical invariants are preserved by computations
Cross-Implementation Validation: Comparison of different implementations of the same mathematical function

Mixed Effects Statistical Models Module Abstract

High-level Purpose and Responsibility

The mixed effects module implements hierarchical statistical models that handle both fixed and random effects for analyzing complex learning data with nested or repeated measures structure. It provides robust statistical inference for multi-level learning experiments while properly accounting for individual differences and clustering in experimental designs.

Key Data Structures and Relationships

MixedEffectsModel: Hierarchical model framework with fixed and random effects components
RandomEffect: Individual-level or group-level random variations in model parameters
FixedEffect: Population-level treatment effects and covariate relationships
CovarianceStructure: Specification of correlation patterns in repeated measures data
HierarchicalData: Multi-level data organization with proper nesting structure
ModelFit: Comprehensive model fitting results with parameter estimates and diagnostics

Main Data Flows and Transformations

Model Specification: Experimental design → Fixed and random effects specification → Model formula construction
Parameter Estimation: Hierarchical data → Maximum likelihood estimation → Fixed and random effect parameters
Inference Procedures: Parameter estimates → Confidence intervals and hypothesis tests → Statistical conclusions
Model Diagnostics: Fitted model → Residual analysis and assumption checking → Model adequacy assessment
Prediction Generation: New data + Fitted model → Conditional predictions → Uncertainty quantification

External Dependencies and Interfaces

Statistics Module: Integration with core statistical functions and hypothesis testing
Experiments Module: Analysis of experimental data with nested or repeated measures designs
Learning Module: Modeling of individual learning trajectories with population-level inferences
Validation Module: Model assumption checking and diagnostic procedures

State Management Patterns

Model State Persistence: Maintains fitted model objects for reuse and prediction
Estimation Algorithm State: Tracks convergence and optimization status during model fitting
Random Effects Prediction: Maintains individual-level random effect estimates
Model Comparison State: Supports comparison of different mixed effects model specifications

Core Algorithms or Business Logic Abstractions

Maximum Likelihood Estimation: Iterative algorithms for parameter estimation in mixed effects models
REML Estimation: Restricted maximum likelihood for unbiased variance component estimation
Random Effects Prediction: Best linear unbiased predictors (BLUPs) for individual-level effects
Likelihood Ratio Testing: Hypothesis testing for fixed and random effects significance
Information Criteria: AIC, BIC for mixed effects model selection and comparison
Bootstrap Inference: Robust confidence intervals and hypothesis tests using resampling methods

Power Analysis Module Abstract

High-level Purpose and Responsibility

The power analysis module provides statistical power calculations and sample size determination for experimental learning studies. It implements power analysis procedures for various statistical tests, enables optimal experimental design through sample size planning, and supports post-hoc power analysis for interpreting experimental results.

Key Data Structures and Relationships

PowerAnalysis: Comprehensive power calculation framework for different statistical tests
SampleSizeCalculator: Optimal sample size determination given effect sizes and power requirements
EffectSizeEstimator: Estimation and specification of expected effect sizes for power calculations
PowerCurve: Visualization and analysis of statistical power across different parameter values
PostHocPower: Retrospective power analysis for completed experiments
OptimalDesign: Integration of power analysis with experimental design optimization

Main Data Flows and Transformations

Effect Size Specification: Domain knowledge → Expected effect sizes → Power calculation inputs
Sample Size Planning: Power requirements + Effect sizes → Optimal sample size recommendations
Power Calculation: Sample sizes + Effect sizes → Statistical power estimates
Design Optimization: Multiple design options → Power comparison → Optimal design selection
Post-Hoc Analysis: Completed experiment data → Retrospective power assessment → Result interpretation

External Dependencies and Interfaces

Statistics Module: Integration with statistical tests and effect size calculations
Experiments Module: Power-informed experimental design and sample size planning
Learning Module: Effect size estimation for learning interventions and treatments
Validation Module: Validation of power analysis assumptions and calculations

State Management Patterns

Power Calculation Caching: Performance optimization for repeated power analyses
Effect Size Library: Repository of empirical effect sizes for different learning interventions
Design History Tracking: Maintains records of power analysis decisions for experimental designs
Sensitivity Analysis State: Tracking of power sensitivity to assumption violations

Core Algorithms or Business Logic Abstractions

Parametric Power Calculations: Analytical power formulas for t-tests, ANOVA, and regression
Non-Parametric Power: Bootstrap and simulation-based power analysis for non-parametric tests
Bayesian Power Analysis: Power calculations for Bayesian experimental designs
Adaptive Power Monitoring: Sequential power analysis for adaptive experimental designs
Multi-Objective Power Optimization: Balancing statistical power across multiple outcome measures
Sensitivity Analysis: Assessment of power robustness to assumption violations and parameter uncertainty

Performance Prediction Module Abstract

High-level Purpose and Responsibility

The performance prediction module implements mathematical models for forecasting learning performance and optimization schedules based on empirical learning curves. It provides curve-fitting algorithms, trajectory analysis, and schedule optimization to predict future learning outcomes and optimize training interventions.

Key Data Structures and Relationships

PerformanceCurve: Mathematical models representing learning performance over time or practice
PowerLawModel: Power law learning curves with asymptotic performance predictions
ExponentialModel: Exponential approach to mastery with rate parameter estimation
LogisticModel: S-shaped learning curves with initial learning phase and plateau
BiExponentialModel: Dual-phase learning with distinct fast and slow learning components
CurveParameters: Fitted parameters for learning curve models with uncertainty estimates
PredictionInterval: Confidence and prediction intervals for future performance forecasts

Main Data Flows and Transformations

Data Preparation: Learning performance history → Time series organization → Model fitting preparation
Curve Fitting: Performance data → Parameter estimation → Fitted learning curve models
Model Selection: Multiple curve fits → Information criteria comparison → Optimal model selection
Performance Forecasting: Fitted models + Future time points → Performance predictions + Uncertainty
Schedule Optimization: Performance predictions → Optimal training schedules → Intervention timing

External Dependencies and Interfaces

Statistics Module: Parameter estimation, model fitting, and statistical inference procedures
Learning Module: Integration with learner performance data and proficiency tracking
Experiments Module: Predictive modeling for experimental outcome forecasting
Validation Module: Model validation and goodness-of-fit assessment

State Management Patterns

Fitted Model Storage: Maintains fitted curve parameters for prediction and analysis
Prediction Cache: Performance optimization for repeated predictions with same parameters
Model Comparison State: Tracks model selection results and performance comparisons
Parameter Uncertainty Tracking: Maintains uncertainty estimates for robust prediction

Core Algorithms or Business Logic Abstractions

Non-Linear Curve Fitting: Levenberg-Marquardt and other optimization algorithms for parameter estimation
Model Selection Criteria: AIC, BIC, and cross-validation for learning curve model comparison
Extrapolation Methods: Robust prediction beyond observed data with uncertainty quantification
Multi-Model Ensemble: Combining predictions from multiple curve models for improved accuracy
Schedule Optimization: Mathematical optimization of training schedules based on predicted learning curves
Change Point Detection: Identification of transitions between different learning phases or regimes

Statistical Validation Module Abstract

High-level Purpose and Responsibility

The statistical validation module provides comprehensive validation procedures for statistical models and assumptions underlying learning experiments. It implements assumption checking, model diagnostics, cross-validation procedures, and robustness testing to ensure the reliability and validity of statistical inferences in learning research.

Key Data Structures and Relationships

ValidationSuite: Comprehensive collection of validation tests for statistical procedures
AssumptionTest: Specific tests for statistical assumptions (normality, homoscedasticity, independence)
ModelDiagnostic: Diagnostic procedures for assessing model fit and identifying issues
CrossValidation: Framework for model validation using data partitioning strategies
RobustnessAnalysis: Sensitivity analysis for statistical procedures under assumption violations
ValidationReport: Comprehensive summary of validation results with recommendations

Main Data Flows and Transformations

Assumption Testing: Statistical model → Assumption validation → Diagnostic results
Model Validation: Fitted models → Cross-validation procedures → Performance assessment
Robustness Analysis: Statistical procedures → Sensitivity testing → Robustness evaluation
Diagnostic Computation: Model residuals → Diagnostic plots and statistics → Model adequacy assessment
Validation Integration: Multiple validation tests → Comprehensive reports → Model acceptance decisions

External Dependencies and Interfaces

Statistics Module: Integration with core statistical functions and hypothesis testing procedures
Learning Module: Validation of learning models and proficiency estimation procedures
Experiments Module: Validation of experimental designs and outcome analyses
Data Module: Access to raw data for assumption checking and validation procedures

State Management Patterns

Validation State Tracking: Maintains validation status for different statistical procedures
Assumption Violation Logging: Records assumption violations and their potential impact
Model Adequacy Assessment: Tracks model fit quality and diagnostic results
Validation History: Maintains records of validation procedures for reproducibility

Core Algorithms or Business Logic Abstractions

Normality Testing: Shapiro-Wilk, Kolmogorov-Smirnov, and other normality assessment procedures
Homoscedasticity Testing: Levene's test, Bartlett's test for equality of variances
Independence Assessment: Durbin-Watson test, autocorrelation analysis for temporal dependencies
Outlier Detection: Statistical and robust methods for identifying anomalous observations
Model Selection Criteria: AIC, BIC, and other information criteria for model comparison
Cross-Validation Strategies: k-fold, leave-one-out, and stratified validation procedures

Data Module - Abstract Documentation

Purpose and Responsibility

The data module serves as the central hub for all data collection, processing, and export functionality in the research system. It orchestrates multiple data streams including audio recordings, user interactions, sensor data, performance metrics, and comprehensive data export capabilities.

Key Data Structures and Relationships

Module Organization

audio_recording: Think-aloud protocol recording and analysis
export: Comprehensive data export and statistical analysis preparation
interaction_tracking: Detailed user interaction pattern capturing
performance_tracing: System performance monitoring and optimization
sensor_integration: External physiological sensor data collection

Core Data Flow

Collection Layer: Multiple concurrent data streams (audio, interactions, sensors, performance)
Processing Layer: Real-time analysis, quality assessment, and pattern detection
Export Layer: Structured data transformation for analysis tools (R, Python, SPSS)

Main Data Flows and Transformations

Input Streams

Audio data from microphone systems across platforms
Mouse/keyboard interaction events with timing precision
External sensor readings (EEG, GSR, eye-tracking)
System performance metrics and timing data

Processing Pipeline

Real-time Collection: Concurrent data streams with timestamp synchronization
Quality Assessment: Signal quality, data completeness, error detection
Pattern Analysis: Behavioral pattern detection, cognitive load estimation
Data Validation: Integrity checks, outlier detection, missing data handling

Output Formats

JSON for structured data interchange
CSV for statistical analysis software
Platform-specific analysis scripts (R, Python, SPSS)
Real-time metrics for live monitoring

External Dependencies and Interfaces

Platform Integration

Audio Systems: Core Audio (macOS), AVAudioRecorder (iOS), MediaRecorder (Android)
Sensor APIs: Device-specific SDKs for EEG, GSR, eye-tracking equipment
Performance Monitoring: System tracing APIs and performance counters

Analysis Tool Compatibility

R Integration: Mixed-effects models, learning curve analysis, group comparisons
Python Integration: Scientific computing stack (pandas, numpy, scipy)
SPSS Integration: Syntax generation for statistical procedures

State Management Patterns

Session-Based Architecture

Session Lifecycle: Start → Active Recording → Quality Assessment → Export
Multi-Stream Coordination: Synchronized timestamps across all data sources
Buffer Management: Circular buffers for real-time data with configurable retention

Data Integrity

Atomic Operations: Complete session recording or rollback
Error Recovery: Graceful handling of sensor failures or data corruption
Privacy Controls: Configurable data retention and anonymization policies

Core Algorithms and Business Logic Abstractions

Audio Analysis

Think-Aloud Classification: NLP-based categorization of cognitive processes
Speech Quality Assessment: Signal-to-noise ratio, clipping detection, silence analysis
Real-time Transcription: Automated speech recognition with confidence scoring

Interaction Pattern Analysis

Keystroke Dynamics: Inter-key intervals, dwell times, correction patterns
Mouse Movement Profiling: Velocity, acceleration, trajectory smoothness
Hesitation Detection: Pause analysis, behavioral uncertainty markers

Sensor Data Processing

Multi-Modal Synchronization: Timestamp alignment across sensor modalities
Artifact Detection: Automated identification of sensor noise and movement artifacts
Feature Extraction: Time-domain and frequency-domain signal characteristics

Export and Analysis Preparation

Population Analytics: Cross-participant comparisons and group statistics
Learning Curve Generation: Progress tracking and performance trajectory analysis
Statistical Modeling: Data formatting for mixed-effects models and hypothesis testing

Performance Considerations

Concurrent Processing: Multi-threaded data collection with minimal interference
Memory Management: Efficient buffering strategies for long recording sessions
Disk I/O Optimization: Asynchronous writing and compression for large datasets
Real-time Constraints: Sub-millisecond timing accuracy for time-sensitive measurements

Security and Privacy Implications

Data Anonymization: Configurable PII removal and participant ID obfuscation
Consent Management: Fine-grained control over data collection and retention
Secure Storage: Encrypted data at rest with configurable retention policies
Access Controls: Role-based permissions for data export and analysis

Interaction Tracking Module - Abstract Documentation

Purpose and Responsibility

Captures comprehensive user interaction patterns including keystroke dynamics, mouse movements, focus events, and behavioral analysis. Provides fine-grained temporal analysis of user behavior for cognitive load assessment and usability research.

Key Data Structures and Relationships

Core Tracking Architecture

InteractionSession: Complete interaction recording session with multi-modal data streams
KeystrokeEvent: Individual key events with timing, context, and correction analysis
MouseEvent: Mouse movements, clicks, and gestures with trajectory analysis
BehavioralPattern: High-level interaction patterns and cognitive load indicators

Interaction Taxonomy

KeyEventType: KeyDown, KeyUp, KeyPress with dwell time measurement
MouseEventType: Move, Click, DoubleClick, Scroll, Hover with velocity tracking
CorrectionType: Backspace, Delete, Cut, Overwrite, SelectAndReplace analysis
HesitationType: Pause classification and uncertainty detection

Context Integration

KeystrokeContext: Task association, input field identification, cursor position tracking
MouseContext: Target elements, click precision, movement efficiency
FocusEvents: Attention switching and window management behavior

Main Data Flows and Transformations

Real-time Collection Pipeline

Event Capture: Low-level input event interception and timestamp recording
Context Enrichment: UI element identification and task state correlation
Feature Extraction: Timing intervals, movement metrics, correction patterns
Quality Assessment: Event sequence validation and missing data detection

Analysis Pipeline

Temporal Analysis: Inter-event intervals, dwell times, pause classification
Spatial Analysis: Mouse trajectory smoothness, click accuracy, movement efficiency
Behavioral Classification: Hesitation detection, correction pattern analysis
Cognitive Load Estimation: Complexity indicators from interaction patterns

Pattern Recognition

Typing Dynamics: Individual biometric signatures from keystroke patterns
Hesitation Analysis: Pause detection and uncertainty quantification
Error Correction: Mistake identification and recovery strategy analysis
Attention Patterns: Focus switching and multitasking behavior

External Dependencies and Interfaces

Platform Input Systems

Operating System: Native input event hooks and accessibility APIs
Web Browser: JavaScript event handling and DOM interaction tracking
Mobile Platforms: Touch event processing and gesture recognition
Cross-Platform: Unified interface across different input modalities

UI Integration

Element Identification: Target UI component recognition and classification
Layout Awareness: Spatial relationship understanding for context analysis
State Synchronization: UI state correlation with interaction events

State Management Patterns

Session Lifecycle Management

Idle → Recording Started → Active Tracking → Analysis Processing → Export Ready

Real-time Processing

Event Buffer: Circular buffer for low-latency event processing
Context Cache: UI state caching for efficient context lookup
Pattern Accumulator: Incremental pattern detection and update

Quality Control

Event Validation: Temporal consistency and logical sequence checking
Missing Data: Gap detection and interpolation strategies
Noise Filtering: Spurious event removal and signal cleaning

Core Algorithms and Business Logic Abstractions

Keystroke Analysis

Dwell Time Calculation: Key press duration measurement and normalization
Inter-Key Intervals: Flight time between keystrokes for typing rhythm analysis
Correction Detection: Edit distance analysis and error recovery patterns
Typing Speed: Words per minute with accuracy adjustment

Mouse Movement Analysis

Trajectory Analysis: Path smoothness, directional changes, sub-movements
Velocity Profiling: Speed and acceleration patterns for motor control analysis
Target Acquisition: Click accuracy and approach strategy assessment
Gesture Recognition: Complex mouse patterns and interaction sequences

Behavioral Pattern Detection

Hesitation Identification: Pause classification and uncertainty markers
Cognitive Load Indicators: Interaction complexity and mental effort estimation
Attention Analysis: Focus duration and switching frequency measurement
Error Recovery: Mistake correction strategies and efficiency metrics

Temporal Dynamics

Rhythm Analysis: Periodic patterns in typing and interaction behavior
Micro-Pause Detection: Brief hesitations and processing delays
Flow State Indicators: Sustained interaction periods with minimal interruption
Fatigue Detection: Performance degradation patterns over time

Performance Considerations

Low-Latency Capture: Minimal overhead for real-time interaction recording
Memory Efficiency: Streaming processing with bounded memory usage
CPU Optimization: Efficient event processing without UI interference
Storage Optimization: Compressed event streams with configurable retention

Privacy and Security Implications

Sensitive Data: Keystroke content filtering and secure disposal
Biometric Privacy: Typing dynamics as personal identifiers
Screen Content: UI element identification without content recording
Consent Management: Granular permission control for different interaction types

Research Applications

Usability Testing: Interaction efficiency and user experience measurement
Cognitive Assessment: Mental workload and processing demand estimation
Behavioral Analysis: Individual differences in interaction strategies
Accessibility Research: Motor control and assistive technology evaluation

Experimental Integration

Task Synchronization: Interaction events correlated with experimental stimuli
Multi-modal Fusion: Integration with eye-tracking, physiological, and audio data
Real-time Feedback: Live interaction quality indicators for participant guidance
Longitudinal Analysis: Learning and adaptation patterns over extended sessions

Performance Tracing Module - Abstract Documentation

Purpose and Responsibility

Provides comprehensive system performance monitoring and optimization capabilities for research applications. Tracks operation durations, identifies performance bottlenecks, and enables data-driven performance optimization with structured logging integration.

Key Data Structures and Relationships

Core Performance Architecture

PerformanceMetrics: Individual operation timing and context data
PerformanceTracker: Active operation monitoring with automatic cleanup
Track Performance Macro: Convenient API for operation timing with minimal overhead

Context Integration

Operation Identification: Hierarchical operation naming and categorization
Thread Tracking: Multi-threaded operation correlation and analysis
Additional Context: Flexible key-value metadata attachment for analysis

Logging Integration

Tracing Spans: Structured logging with hierarchical operation tracking
Performance Thresholds: Automatic log level adjustment based on operation duration
Debug Information: Detailed timing information for development and optimization

Main Data Flows and Transformations

Performance Tracking Pipeline

Operation Start: Timestamp recording and context initialization
Context Enrichment: Dynamic metadata attachment during operation
Duration Measurement: High-resolution timing with thread identification
Threshold Classification: Automatic categorization based on performance characteristics

Analysis Pipeline

Performance Classification: Fast (0-100ms), Normal (101-1000ms), Slow (1-5s), Very Slow (5s+)
Trend Analysis: Operation performance tracking over time
Bottleneck Identification: Systematic slow operation detection
Resource Correlation: Performance correlation with system resource usage

Logging Integration

Structured Logging: Integration with tracing framework for hierarchical analysis
Automatic Alerting: Performance threshold-based warning generation
Debug Support: Detailed timing information for development workflows

External Dependencies and Interfaces

System Integration

High-Resolution Timing: System clock access for microsecond precision
Thread Identification: Operating system thread tracking and correlation
Resource Monitoring: CPU, memory, and I/O performance correlation

Tracing Framework

Span Hierarchy: Nested operation tracking with parent-child relationships
Metadata Attachment: Flexible attribute system for operation context
Log Level Management: Dynamic logging based on performance characteristics

State Management Patterns

Tracker Lifecycle

Created → Context Addition → Operation Execution → Duration Measurement → Metrics Generation

Resource Management

RAII Pattern: Automatic cleanup with Rust ownership semantics
Memory Efficiency: Minimal overhead during operation execution
Thread Safety: Concurrent operation tracking without synchronization overhead

Error Handling

Graceful Degradation: Performance tracking failure doesn't affect operation execution
Missing Context: Safe handling of incomplete performance data
Clock Issues: Fallback strategies for timing inconsistencies

Core Algorithms and Business Logic Abstractions

Timing Measurement

High-Resolution Clocks: Nanosecond precision timing for accurate measurement
Clock Stability: Monotonic clocks for reliable interval measurement
Overhead Compensation: Measurement infrastructure overhead accounting

Performance Classification

Threshold-Based Categorization: Dynamic performance bucket assignment
Context-Aware Analysis: Operation type consideration in performance evaluation
Statistical Analysis: Performance distribution analysis and outlier detection

Optimization Support

Hotspot Identification: Automatic detection of performance-critical operations
Regression Detection: Performance degradation identification over time
Resource Correlation: Performance bottleneck root cause analysis

Performance Considerations

Minimal Overhead: Sub-microsecond tracking infrastructure impact
Memory Footprint: Bounded memory usage for long-running operations
CPU Efficiency: Optimized timing code with minimal instruction overhead
Scalability: Support for high-frequency operation tracking

Development and Debugging Support

Development Workflow

Performance-Driven Development: Built-in performance measurement for optimization
Regression Testing: Automated performance regression detection
Profiling Integration: Structured performance data for external profiling tools

Production Monitoring

Real-time Performance: Live operation performance monitoring
Performance Alerting: Automatic notification of performance anomalies
Historical Analysis: Long-term performance trend analysis and capacity planning

Macro System

Ergonomic API: Simple macro interface for common performance tracking patterns
Zero-Cost Abstractions: Compile-time optimization for minimal runtime impact
Flexible Usage: Block-based timing and manual tracker management options

Integration Patterns

Research Application Support

Experimental Timing: Precise timing measurement for cognitive experiments
Response Time Analysis: User interaction timing with research-grade accuracy
System Load Correlation: Performance impact analysis on experimental validity

Timestamp Synchronization: Coordinated timing across multiple data streams
Performance Context: System performance as experimental variable
Real-time Feedback: Performance-based experimental adaptation

Quality Assurance

Performance Baselines: Expected performance ranges for operation validation
Automated Testing: Performance regression detection in CI/CD pipelines
Resource Usage: Performance correlation with system resource consumption

Audio Recording Module - Abstract Documentation

Purpose and Responsibility

Implements comprehensive think-aloud protocol recording and analysis for research applications. Provides cross-platform audio capture, real-time quality assessment, automatic transcription, and cognitive process classification from verbal protocols.

Key Data Structures and Relationships

Core Architecture

AudioRecorder: Main recording controller with platform-specific backends
AudioSession: Complete recording session with metadata and analysis
ThinkAloudAnalyzer: NLP-based cognitive process classifier
AudioFile: Individual recording file with quality metrics

Think-Aloud Taxonomy

ThinkAloudType: Planning, Execution, Monitoring, Evaluation, Struggle, Insight, Metacognition
EmotionalMarker: Frustration, Confidence, Confusion, Satisfaction with intensity scoring
CognitiveProcess: Problem-solving, pattern recognition, working memory load analysis

Main Data Flows and Transformations

Recording Pipeline

Session Initialization: Participant setup, device configuration, consent verification
Real-time Capture: Platform-specific audio recording with quality monitoring
Buffer Management: Circular buffering with configurable retention and overflow handling
Quality Assessment: SNR calculation, clipping detection, silence percentage analysis

Analysis Pipeline

Transcription: Automatic speech recognition with confidence scoring
Segmentation: Think-aloud segment identification and temporal boundary detection
Classification: NLP pattern matching for cognitive process identification
Emotion Analysis: Sentiment analysis and emotional marker extraction

Export Pipeline

Raw Audio: Platform-specific format preservation (WAV, MP3, AAC, FLAC)
Structured Transcripts: Time-aligned text with speaker identification
Analysis Metadata: Cognitive classifications, emotional markers, quality metrics

External Dependencies and Interfaces

Platform Audio Systems

macOS: Core Audio framework integration
iOS: AVAudioRecorder with iOS-specific permissions
Android: MediaRecorder with Android audio subsystem
Generic: Cross-platform fallback implementation

Real-time Processing

Threading: Separate recording thread for minimal latency
Memory Management: Lock-free circular buffers for audio samples
Signal Processing: RMS calculation, frequency analysis, noise detection

State Management Patterns

Session State Machine

Idle → Session Started → Recording Active → Recording Stopped → Analysis Complete → Export Ready

Concurrent Operations

Recording Thread: High-priority audio capture with minimal processing
Analysis Thread: Background processing for transcription and classification
Quality Monitor: Real-time metrics calculation without recording interruption

Error Recovery

Device Failure: Graceful degradation and error reporting
Buffer Overflow: Oldest data eviction with preservation of recent samples
Processing Errors: Fallback to raw audio preservation without analysis

Core Algorithms and Business Logic Abstractions

Audio Quality Assessment

Signal-to-Noise Ratio: Real-time SNR calculation for recording quality
Clipping Detection: Amplitude threshold monitoring for overload prevention
Silence Analysis: Adaptive threshold detection for speech activity
Overall Quality Scoring: Composite metric combining multiple quality factors

Think-Aloud Analysis

Pattern Matching: Keyword-based classification with confidence scoring
Temporal Segmentation: Pause-based boundary detection for think-aloud units
Cognitive Process Mapping: Evidence-based classification of mental operations
Emotional State Inference: Linguistic markers of affective states

Real-time Metrics

Volume Monitoring: Current audio levels for recording guidance
Buffer Status: Memory usage and recording duration tracking
Quality Indicators: Live feedback for recording optimization

Performance Considerations

Low-Latency Recording: Minimal processing delay for natural interaction
Memory Efficiency: Streaming processing without full audio buffering
CPU Usage: Optimized signal processing for real-time operation
Battery Life: Power-efficient recording for mobile platforms

Privacy and Compliance

Informed Consent: Configurable consent recording and verification
Data Retention: Automatic deletion policies with participant control
Anonymization: Speaker identification removal and transcript sanitization
GDPR Compliance: Data subject rights and processing transparency

Research Integration

Multi-modal Synchronization: Timestamp coordination with other data streams
Experimental Control: Recording triggers aligned with task presentation
Analysis Export: Integration with statistical software and research workflows
Validity Measures: Inter-rater reliability and classification accuracy metrics

Sensor Integration Module - Abstract Documentation

Purpose and Responsibility

Provides comprehensive integration framework for external physiological and biometric sensors including EEG, GSR, eye-tracking, heart rate, and other research-grade measurement devices. Enables multi-modal data collection with precise timestamp synchronization and real-time quality monitoring.

Key Data Structures and Relationships

Core Sensor Architecture

SensorSession: Complete multi-sensor recording session with synchronized data streams
SensorConfig: Individual sensor configuration and calibration parameters
SensorReading: Individual sensor measurements with timestamp and quality metrics
DataQuality: Real-time assessment of signal quality and measurement reliability

Sensor Type Hierarchy

EEG: Multi-channel brain activity recording with configurable montages
GSR: Galvanic skin response for autonomic arousal measurement
EyeTracking: Gaze position and pupil diameter monitoring
HeartRate: Cardiac rhythm and heart rate variability analysis
Environmental: Temperature, accelerometer, blood pressure monitoring

Synchronization Framework

SyncEvent: Cross-modal timestamp anchoring for data alignment
CalibrationData: Sensor-specific calibration parameters and drift correction
ConnectionType: USB, Bluetooth, WiFi, and custom protocol support

Main Data Flows and Transformations

Data Collection Pipeline

Sensor Discovery: Automatic detection and configuration of available sensors
Calibration Process: Individual sensor calibration and baseline establishment
Synchronized Recording: Multi-modal data collection with microsecond timestamp precision
Quality Monitoring: Real-time signal quality assessment and artifact detection

Processing Pipeline

Artifact Detection: Automatic identification of movement artifacts and signal noise
Signal Filtering: Real-time digital filtering for noise reduction
Feature Extraction: Time-domain and frequency-domain feature computation
Data Validation: Missing data detection and quality assessment

Export Pipeline

Raw Data: Unprocessed sensor streams with full temporal resolution
Processed Features: Computed metrics and derived measurements
Synchronized Datasets: Multi-modal data aligned to common timeline
Quality Reports: Comprehensive data quality assessment and validation

External Dependencies and Interfaces

Hardware Integration

EEG Systems: Biosemi, Brain Products, Emotiv, OpenBCI compatibility
Eye Trackers: Tobii, SR Research, Pupil Labs integration
GSR Devices: Biopac, iMotions, Shimmer sensor support
Custom Protocols: Extensible framework for proprietary sensor systems

Communication Protocols

USB Integration: High-speed data transfer with low latency
Wireless Protocols: Bluetooth LE, WiFi, and custom RF communication
Network Streaming: Real-time data streaming over TCP/IP and UDP
Serial Communication: RS-232 and custom serial protocol support

Platform Support

Cross-Platform: Windows, macOS, Linux sensor driver integration
Real-time Systems: Low-latency data collection with priority scheduling
Mobile Platforms: iOS and Android sensor integration where applicable

State Management Patterns

Sensor Lifecycle Management

Discovery → Configuration → Calibration → Recording → Quality Assessment → Export

Multi-Sensor Coordination

Synchronization: Master clock coordination across multiple sensor streams
Error Recovery: Graceful handling of individual sensor failures
Resource Management: CPU and bandwidth allocation across sensor channels

Data Integrity

Buffer Management: Circular buffers with overflow protection
Timestamp Validation: Clock synchronization and drift correction
Missing Data: Gap detection and interpolation strategies

Core Algorithms and Business Logic Abstractions

Signal Processing

Digital Filtering: Real-time IIR and FIR filter implementation
Artifact Rejection: Statistical and template-based artifact identification
Baseline Correction: Drift correction and baseline restoration
Frequency Analysis: Real-time spectral analysis and band power computation

EEG-Specific Processing

Channel Montaging: Flexible electrode configuration and re-referencing
Impedance Monitoring: Real-time electrode contact quality assessment
Event-Related Analysis: Stimulus-locked averaging and time-frequency analysis
Connectivity Analysis: Coherence and phase synchronization measurement

Eye-Tracking Analysis

Gaze Estimation: Pupil detection and gaze vector computation
Saccade Detection: Rapid eye movement identification and classification
Fixation Analysis: Stable gaze period detection and duration measurement
Pupil Response: Pupil diameter changes and cognitive load estimation

Physiological Monitoring

Heart Rate Variability: R-R interval analysis and autonomic assessment
GSR Analysis: Skin conductance level and response detection
Movement Analysis: Accelerometer data processing for motion artifacts
Environmental Correlation: Temperature and environmental factor integration

Performance Considerations

Real-time Processing: Sub-millisecond latency for time-critical applications
High-Throughput: Support for high sampling rates (up to 20kHz+) across multiple channels
Memory Management: Efficient buffering for long recording sessions
CPU Optimization: Multi-threaded processing with SIMD acceleration where applicable

Quality Assurance and Validation

Data Quality Metrics

Signal-to-Noise Ratio: Real-time SNR calculation and monitoring
Artifact Contamination: Percentage of data affected by artifacts
Missing Data: Temporal gaps and discontinuities in sensor streams
Calibration Drift: Long-term sensor stability and recalibration needs

Validation Protocols

Cross-Modal Validation: Consistency checking across sensor modalities
Known Signal Testing: Validation with synthetic and calibration signals
Inter-Device Reliability: Consistency across multiple sensor units
Temporal Validation: Synchronization accuracy verification

Research Integration and Applications

Experimental Design Support

Event Triggering: Synchronized stimulus presentation and sensor recording
Condition Marking: Experimental condition annotation in sensor streams
Real-time Feedback: Live biofeedback and neurofeedback applications
Multi-Session Studies: Longitudinal data collection and analysis

Analysis Integration

Statistical Software: Export compatibility with MATLAB, R, Python
Specialized Tools: Integration with EEGLAB, FieldTrip, MNE-Python
Machine Learning: Feature extraction for classification and prediction
Visualization: Real-time and offline data visualization tools

Privacy and Ethics Considerations

Biometric Privacy: Secure handling of physiological identifiers
Informed Consent: Comprehensive consent for invasive monitoring
Data Anonymization: Biometric feature removal and participant protection
Medical Compliance: Healthcare data handling and HIPAA considerations where applicable

Data Export Module - Abstract Documentation

Purpose and Responsibility

Provides comprehensive data export and statistical analysis preparation functionality. Transforms learning session data into analysis-ready formats for R, Python, SPSS, and other statistical software packages, with automated script generation and population-level analytics.

Key Data Structures and Relationships

Core Export Architecture

LearnerDataExport: Complete participant dataset with sessions, performance, and model parameters
PopulationAnalyzer: Cross-participant analysis and group comparison functionality
SessionExporter: Individual session data formatting and summary generation

Data Hierarchy

Population → Participants → Sessions → Trials → Responses

Analysis Structure

PerformancePoint: Learning trajectory data points with accuracy and timing
ErrorAnalysis: Systematic error pattern identification and confusion matrices
ModelSnapshot: Cognitive model parameters at specific time points
SessionSummary: Aggregated performance metrics per session

Main Data Flows and Transformations

Individual Export Pipeline

Data Aggregation: Session history, model parameters, performance trajectories
Quality Assessment: Data completeness, outlier detection, validation checks
Format Conversion: JSON, CSV, and platform-specific formats
Script Generation: Automated analysis code for target platforms

Population Export Pipeline

Data Consolidation: Multi-participant dataset assembly
Group Analysis: Between-subjects comparisons and statistical testing
Norm Generation: Population-based difficulty metrics and item analysis
Collaborative Export: Comprehensive analysis packages with documentation

Statistical Analysis Preparation

Mixed-Effects Modeling: Hierarchical data structure for multilevel analysis
Learning Curve Analysis: Time-series data with individual and group trajectories
Transfer Testing: Phase-based performance comparison and gain scoring
Strategy Classification: Behavioral pattern identification and clustering

External Dependencies and Interfaces

Statistical Software Integration

R Integration: tidyverse, lme4, lmerTest compatibility with automated scripts
Python Integration: pandas, scipy, statsmodels compatibility with Jupyter notebooks
SPSS Integration: Syntax generation and data import procedures

Export Format Support

Structured Data: JSON for programmatic access and API integration
Tabular Data: CSV with research-standard variable naming conventions
Analysis Scripts: Generated code with embedded documentation and interpretation
Codebooks: Human-readable variable descriptions and analysis guidance

State Management Patterns

Export Session Management

Incremental Export: Progressive data export without complete session reprocessing
Version Control: Export timestamp tracking and data provenance
Error Recovery: Partial export completion with resume capability

Data Integrity

Validation Pipeline: Multi-level data quality checks and missing data handling
Consistency Verification: Cross-session validation and participant matching
Privacy Preservation: Automatic anonymization and PII removal

Core Algorithms and Business Logic Abstractions

Performance Analysis

Learning Rate Estimation: Individual trajectory fitting and slope calculation
Strategy Detection: Response pattern analysis and behavioral classification
Error Pattern Mining: Systematic mistake identification and categorization
Difficulty Calibration: Item response theory and empirical difficulty estimation

Statistical Preparation

Data Transformation: Long/wide format conversion for analysis requirements
Variable Engineering: Derived measures and composite scores
Group Assignment: Experimental condition inference and validation
Phase Detection: Training/test/transfer phase identification

Population Analytics

Group Comparisons: Effect size calculation and statistical significance testing
Learning Curve Analysis: Multi-level modeling with individual and group trajectories
Transfer Analysis: Cross-phase performance comparison and retention measurement
Individual Differences: Cognitive profile analysis and strategy clustering

Quality Assurance

Data Completeness: Missing data pattern analysis and reporting
Outlier Detection: Statistical and domain-based anomaly identification
Validity Checks: Logical consistency and temporal ordering validation

Performance Considerations

Memory Efficiency: Streaming export for large datasets without full memory loading
Processing Speed: Optimized algorithms for population-scale data processing
Disk I/O: Efficient file writing with compression for large exports
Scalability: Population-level analysis capability for research consortium data

Research Methodology Support

Experimental Design Integration

Between-Subjects Analysis: Group comparison and randomization validation
Within-Subjects Analysis: Repeated measures and learning progression
Mixed Design: Combined between/within factors with proper error terms

Statistical Analysis Support

Power Analysis: Sample size estimation and effect size reporting
Multiple Comparisons: Family-wise error correction and post-hoc testing
Model Selection: Automated model comparison and fit statistics
Assumption Testing: Normality, homoscedasticity, and independence verification

Publication Ready Output

APA Format: Statistical reporting aligned with publication standards
Effect Sizes: Cohen's d, eta-squared, and confidence interval reporting
Visualization: Publication-quality plots with appropriate statistical annotations
Reproducibility: Complete analysis scripts with version control and documentation

Data Privacy and Ethics

Anonymization: Systematic PII removal with participant ID scrambling
Consent Tracking: Data use permission verification and scope limitation
Data Minimization: Export scope limitation to research-necessary variables
Retention Policies: Automatic data lifecycle management and deletion scheduling

compliance/mod.rs - Compliance Module Root Abstract

High-Level Purpose

Module root that exposes the compliance subsystem for research ethics, regulatory adherence, and scientific integrity within the abcdeez-core framework.

Key Data Structures and Relationships

Module Organization: Simple module aggregation exposing four primary compliance domains:
- audit_trail: Comprehensive audit logging and event tracking
- citation_manager: Academic citation management and methodology documentation
- irb: Institutional Review Board compliance and ethics documentation generation
- preregistration: Study pre-registration and analysis plan validation

Main Data Flows

Unified Access: Provides centralized access point for all compliance-related functionality
Domain Separation: Clean separation between different aspects of research compliance

External Dependencies

Standard Rust module system only

State Management Patterns

Stateless Organization: Pure module aggregation without internal state

Core Abstractions

Compliance Integration: Unified interface for diverse compliance requirements
Regulatory Coverage: Comprehensive coverage of research ethics and integrity requirements

compliance/audit_trail.rs - Audit Trail Management System Abstract

High-Level Purpose

Comprehensive audit trail management system for research compliance, providing detailed logging, tracking, and reporting of all system activities, user interactions, and data modifications with configurable audit levels and retention policies.

Key Data Structures and Relationships

AuditTrailManager: Central audit management with configurable settings
AuditEvent: Immutable event records with comprehensive metadata
Actor/Resource/Operation Model: Fine-grained action tracking with structured relationships
Configuration Hierarchy: Flexible audit levels (Minimal → Standard → Detailed → Forensic)
Event Classification: Structured event types (Authentication, Data Access, System Events, etc.)

Main Data Flows

Event Collection: Real-time capture of system activities with structured metadata
Session Management: Session-based event correlation and tracking
Filtering Pipeline: Configurable event filtering based on audit level settings
Report Generation: Automated compliance report generation with statistical summaries
Data Export: Structured export capabilities for external compliance systems

External Dependencies

chrono: Timestamp management and date/time operations
serde: Serialization for audit record persistence and export
tracing: Structured logging integration with audit events
uuid: Unique identifier generation for audit event tracking

State Management Patterns

Event Accumulation: Append-only event storage with immutable records
Session Context: Maintained session state for event correlation
Configuration State: Runtime configuration management for audit behavior
Retention Management: Automated cleanup based on configurable retention policies

Core Algorithms and Business Logic Abstractions

Event Classification: Automatic categorization of events by type and importance
Level-Based Filtering: Dynamic filtering based on configured audit granularity
Integrity Verification: Event integrity checking through checksums and validation
Compliance Reporting: Statistical analysis and summary generation for regulatory reporting
Data Anonymization: Privacy-preserving audit trails with configurable anonymization

Security and Compliance Features

Immutable Records: Tamper-evident audit trail with integrity verification
Privacy Controls: Configurable data anonymization and field hashing
Role-Based Tracking: Detailed actor classification and role-based access patterns
Retention Compliance: Automated retention management meeting 7-year research standards

compliance/citation_manager.rs - Academic Citation Management System Abstract

High-Level Purpose

Comprehensive academic citation management system that automates reference handling, methodology documentation, and bibliography generation for research publications, ensuring proper attribution and methodological transparency.

Key Data Structures and Relationships

CitationManager: Central repository for references, methodology citations, and bibliography management
Reference Hierarchy: Structured reference types (Journal, Book, Software, etc.) with complete metadata
Author Management: Comprehensive author information with affiliations and ORCID identifiers
Publication Metadata: Detailed publication information supporting multiple citation formats
Methodology Tracking: Mapping between research methods and supporting citations

Main Data Flows

Reference Import: Automated citation import from DOI and external databases
Method Attribution: Dynamic linking of research methods to supporting literature
Bibliography Generation: Automated formatting in multiple academic styles (APA, Chicago, MLA, etc.)
Methodology Reporting: Comprehensive method documentation with citations and justifications
Export Pipeline: Multi-format export (BibTeX, RIS, EndNote) for integration with manuscript preparation

External Dependencies

serde: Serialization for reference database persistence and export
chrono: Date/time management for publication dates and citation tracking
Standard library collections for reference organization and searching

State Management Patterns

Reference Database: Persistent storage of citation metadata with unique identifier management
Method Tracking: Runtime tracking of used methods for automatic citation generation
Citation History: Maintained history of citation usage and methodology applications
Template Management: Configurable citation templates and formatting rules

Core Algorithms and Business Logic Abstractions

Citation Formatting: Multi-style citation formatting with configurable templates
Method Detection: Automatic detection and citation of methodological approaches
Reference Validation: Citation completeness checking and metadata validation
Search and Discovery: Full-text search across references with relevance ranking
Impact Assessment: Citation counting and reference usage analytics

Academic Integration Features

Standard Compliance: Full compliance with major academic citation styles
Methodological Documentation: Automated methods section generation with proper citations
Transparency Reporting: Comprehensive documentation of analytical approaches and references
Software Citation: Proper attribution of computational tools and statistical software
Default References: Pre-loaded canonical references for common methodological approaches

Publication Workflow Support

Manuscript Integration: Direct export to manuscript preparation systems
Collaboration Support: Shared reference databases with access control
Version Management: Citation versioning and change tracking for collaborative research
Quality Assurance: Automated validation of citation completeness and formatting consistency

compliance/irb.rs - IRB Compliance Documentation Generator Abstract

High-Level Purpose

Comprehensive Institutional Review Board (IRB) compliance system that automatically generates ethics approval documentation, risk assessments, consent forms, and regulatory compliance reports for human subjects research across multiple jurisdictions.

Key Data Structures and Relationships

IRBComplianceGenerator: Central compliance orchestration with institutional context
Risk Assessment Framework: Structured risk evaluation with categorization, mitigation, and monitoring
Consent Management: Multi-template consent form generation with regulatory compliance
Data Protection Planning: Comprehensive data security and privacy compliance framework
Regulatory Adaptation: Multi-jurisdiction support (US, EU, Canada, UK) with appropriate regulatory frameworks

Main Data Flows

Application Generation: Automated IRB application creation from experiment specifications
Risk Analysis Pipeline: Systematic risk identification, assessment, and mitigation planning
Consent Form Creation: Template-based consent form generation with regulatory element compliance
Document Package Export: Complete IRB submission package generation with supporting documentation
Compliance Monitoring: Ongoing compliance status assessment and reporting

External Dependencies

serde: Serialization for compliance documentation and export
chrono: Date/time management for regulatory timelines and compliance tracking
Standard library for file I/O and document generation

State Management Patterns

Institutional Context: Maintained institutional and PI information for application consistency
Risk Registry: Comprehensive risk assessment database with mitigation tracking
Template Library: Consent form templates with population-specific customization
Compliance Status: Real-time compliance monitoring with issue tracking and recommendations

Core Algorithms and Business Logic Abstractions

Risk Stratification: Multi-dimensional risk assessment with probability-severity matrices
Regulatory Mapping: Automatic application of appropriate regulatory frameworks based on jurisdiction
Consent Optimization: Dynamic consent form generation based on study characteristics and populations
Documentation Automation: Intelligent document generation from structured study metadata
Compliance Validation: Comprehensive pre-submission validation and completeness checking

Regulatory Compliance Features

Multi-Jurisdiction Support: Full compliance with US (45 CFR 46), EU (GDPR), and other international standards
Vulnerable Population Protection: Specialized handling of children, prisoners, and other protected populations
Risk Minimization: Systematic risk identification and mitigation planning
Data Protection Integration: Comprehensive data security and privacy compliance frameworks
Audit Trail Integration: Complete documentation lifecycle tracking for regulatory inspection

Ethical Research Support

Systematic Risk Assessment: Evidence-based risk evaluation with standardized methodologies
Transparent Documentation: Complete documentation of ethical considerations and mitigation strategies
Stakeholder Communication: Clear, accessible consent forms and participant communications
Ongoing Monitoring: Continuous compliance monitoring with automated reporting and alerting

compliance/preregistration.rs - Study Pre-registration System Abstract

High-Level Purpose

Comprehensive study pre-registration system that enforces scientific transparency by requiring researchers to specify hypotheses, analysis plans, and decision criteria before data collection, with integrity verification and deviation tracking.

Key Data Structures and Relationships

PreRegistration: Immutable study registration with cryptographic integrity verification
Hypothesis Framework: Structured primary/secondary hypothesis specification with effect predictions
Analysis Plan: Detailed statistical analysis specification with robustness checks and power analysis
Data Collection Protocol: Comprehensive sampling and data quality procedures with stopping rules
Validation System: Real-time analysis validation against pre-registered plans

Main Data Flows

Registration Creation: Structured study specification with validation and completeness checking
Integrity Verification: Cryptographic hashing for tamper-evident registration records
Analysis Validation: Runtime verification that analyses match pre-registered specifications
Deviation Tracking: Systematic documentation of any deviations from original plans
Transparency Reporting: Automated generation of transparency reports for publication

External Dependencies

serde: Serialization for registration persistence and export
chrono: Timestamp management for registration and deviation tracking
sha2: Cryptographic hashing for integrity verification
statrs: Statistical functions for power analysis and sample size calculation

State Management Patterns

Registration Lifecycle: Formal state transitions from Draft → Registered → Analysis Complete
Immutable Records: Tamper-evident registration with cryptographic verification
Deviation Log: Append-only tracking of any deviations with justification and impact assessment
Validation State: Runtime tracking of analysis compliance with pre-registered plans

Core Algorithms and Business Logic Abstractions

Integrity Verification: SHA-256 hashing for registration integrity and tamper detection
Analysis Matching: Variable and method matching between planned and executed analyses
Power Calculation: Statistical power analysis with automatic sample size determination
Transparency Scoring: Comprehensive transparency metrics for research quality assessment
Deviation Analysis: Impact assessment and categorization of protocol deviations

Scientific Integrity Features

Pre-commitment: Binding commitment to analysis plans before seeing data
Hypothesis Registration: Clear distinction between confirmatory and exploratory analyses
Multiple Comparison Control: Built-in correction for multiple testing with pre-specified methods
Effect Size Prediction: Quantitative effect size predictions with confidence intervals
Stopping Rules: Pre-specified criteria for study termination and sample size adjustment

Research Quality Assurance

Complete Documentation: Comprehensive study documentation with methodological details
Reproducible Analysis: Exact specification of all analytical procedures and assumptions
Bias Prevention: Systematic prevention of HARKing (Hypothesizing After Results are Known)
Transparency Enhancement: Public documentation of research plans and modifications
Quality Metrics: Automated calculation of study quality and transparency indicators

Protocol Module - Abstract Documentation

Purpose and Responsibility

Manages experimental protocols, version control, and seed management for reproducible research. Ensures experimental consistency, data integrity, and replicability across research studies.

Key Data Structures and Relationships

Protocol Components

mod: Module organization for protocol management
seed_management: Random number generation control for reproducibility
version_control: Experimental protocol versioning and change tracking
versioning: Data format and analysis pipeline version management

Version Control Architecture

ProtocolVersion → ExperimentalParameters + AnalysisConfiguration + DataSchema
SeedManager → RandomizationControl + ReproducibilityGuarantees

Main Data Flows and Transformations

Protocol Management

Version Creation: New protocol development with change tracking
Seed Control: Deterministic randomization for reproducible experiments
Parameter Validation: Experimental configuration verification
Compatibility Checking: Cross-version compatibility and migration

Reproducibility Pipeline

Seed Generation: Cryptographically secure random seed creation
State Persistence: Complete experimental state serialization
Version Tracking: Protocol evolution and backward compatibility
Audit Trail: Complete history of experimental modifications

Core Algorithms and Business Logic Abstractions

Deterministic Randomization: Reproducible pseudo-random number generation
Version Management: Semantic versioning with compatibility guarantees
Protocol Validation: Configuration consistency and completeness checking
Migration Support: Automatic updating of legacy experimental data

Seed Management Module Abstract

High-level Purpose and Responsibility

The seed management module provides deterministic randomization control for reproducible experiments and learning sessions. It manages random number generator seeds across different system components, enabling exact replication of experimental conditions while supporting controlled randomization for scientific validity and debugging.

Key Data Structures and Relationships

SeedManager: Central coordinator for random seed generation and distribution
SessionSeed: Hierarchical seed structure with master seeds and component-specific derived seeds
SeedHierarchy: Tree-like organization of seeds for different system components and experimental phases
ReproducibilityToken: Unique identifiers linking seeds to specific experimental configurations
SeedHistory: Audit trail of seed usage for experiment reconstruction and debugging
DeterministicRNG: Controlled random number generation with seed-based initialization

Main Data Flows and Transformations

Seed Generation: Experiment initialization → Master seed creation → Component seed derivation
Seed Distribution: Master seeds → Hierarchical decomposition → Component-specific seed assignment
Randomization Control: Seed-based RNG → Deterministic random processes → Reproducible experimental outcomes
Seed Tracking: Seed usage → Historical logging → Reproducibility documentation
Seed Recovery: Experiment identifiers → Seed lookup → Exact experiment reconstruction

External Dependencies and Interfaces

Experiments Module: Seed provision for participant assignment and experimental randomization
Tasks Module: Deterministic task generation and ordering through controlled randomization
Learning Module: Reproducible learning algorithm initialization and random parameter selection
Statistics Module: Controlled random sampling for statistical procedures and validation

State Management Patterns

Immutable Seed Assignment: Seeds remain constant throughout experimental sessions
Hierarchical Seed Derivation: Systematic generation of component seeds from master seeds
Seed Versioning: Integration with protocol versioning for long-term reproducibility
Cross-Session Seed Continuity: Consistent seed management across multi-session experiments

Core Algorithms or Business Logic Abstractions

Cryptographic Seed Generation: Secure random seed creation using system entropy sources
Deterministic Seed Derivation: Reproducible generation of component seeds from master seeds
Seed Space Partitioning: Non-overlapping seed assignment across system components
Reproducibility Validation: Verification that identical seeds produce identical experimental outcomes
Seed Collision Avoidance: Algorithms ensuring unique seed assignment across experiments
Cross-Platform Seed Consistency: Platform-independent seed behavior for multi-environment reproducibility

Protocol Version Control Module Abstract

High-level Purpose and Responsibility

The protocol version control module manages versioning and compatibility checking for experimental protocols and system configurations. It ensures reproducibility across software updates, handles protocol migration, and maintains backward compatibility while enabling systematic evolution of experimental procedures.

Key Data Structures and Relationships

ProtocolVersion: Semantic versioning for experimental protocols with compatibility metadata
VersionHistory: Chronological record of protocol changes and evolution
CompatibilityMatrix: Cross-version compatibility assessment for protocols and data formats
MigrationPath: Systematic procedures for updating protocols and data between versions
ProtocolDiff: Detailed change tracking between protocol versions
VersionValidator: Compatibility checking and validation system for protocol combinations

Main Data Flows and Transformations

Version Assignment: Protocol creation → Semantic version tagging → Version registry update
Compatibility Checking: Protocol combination requests → Compatibility validation → Approval or migration guidance
Protocol Migration: Version updates → Data transformation → Updated protocol compliance
Change Tracking: Protocol modifications → Diff generation → Version history maintenance
Backward Compatibility: Legacy data access → Version-specific handling → Current system integration

External Dependencies and Interfaces

Experiments Module: Protocol versioning for experimental designs and procedures
Learning Module: Algorithm versioning and compatibility checking for learning strategies
Statistics Module: Statistical method versioning and result interpretation consistency
Data Module: Data format versioning and migration support for persistent storage

State Management Patterns

Immutable Version Records: Protocol versions remain unchanged after publication
Incremental Version Evolution: Systematic progression through semantic version numbers
Multi-Version Support: Simultaneous support for multiple protocol versions
Version-Aware Operations: System behavior adapted to specific protocol versions

Core Algorithms or Business Logic Abstractions

Semantic Versioning Logic: Automated version number assignment based on change significance
Compatibility Assessment: Algorithms for determining cross-version compatibility
Automated Migration: Data and protocol transformation procedures for version updates
Change Impact Analysis: Assessment of modification effects on experimental validity
Version Conflict Resolution: Systematic approaches to handling version incompatibilities
Protocol Validation: Verification that protocol changes maintain scientific validity

Versioning System Module Abstract

High-level Purpose and Responsibility

The versioning system module implements semantic versioning standards and comparison algorithms for the experimental learning platform. It provides systematic version number management, compatibility rules, and version ordering logic that supports reproducible science and controlled system evolution.

Key Data Structures and Relationships

Version: Semantic version representation with major, minor, and patch components
VersionRequirement: Specification of acceptable version ranges and constraints
VersionComparison: Algorithms for ordering and comparing version numbers
CompatibilityRule: Logic for determining version compatibility and breaking changes
VersionRange: Specification of acceptable version intervals with inclusion/exclusion criteria
PreReleaseMetadata: Support for development versions and release candidates

Main Data Flows and Transformations

Version Parsing: Version strings → Structured version objects → Validation and normalization
Version Comparison: Version pairs → Ordering relationships → Compatibility decisions
Range Checking: Version + Requirements → Compatibility validation → Acceptance or rejection
Version Ordering: Version collections → Sorted sequences → Latest/optimal version selection
Compatibility Assessment: Version combinations → Rule evaluation → System configuration validation

External Dependencies and Interfaces

Protocol Module: Version comparison for protocol compatibility and migration decisions
Experiments Module: Version validation for experimental configuration consistency
Data Module: Data format versioning and backward compatibility management
Statistics Module: Statistical method versioning and result reproducibility

State Management Patterns

Immutable Version Objects: Version representations remain constant after creation
Cached Comparison Results: Performance optimization for repeated version comparisons
Version Registry Management: Centralized tracking of all system component versions
Validation State Tracking: Maintains validation results for version combinations

Core Algorithms or Business Logic Abstractions

Semantic Version Parsing: Robust parsing of version strings with error handling and normalization
Lexicographic Ordering: Standard version comparison algorithms following semantic versioning rules
Range Satisfaction: Algorithms for testing whether versions satisfy requirement specifications
Breaking Change Detection: Identification of version changes that break backward compatibility
Version Resolution: Algorithms for finding optimal version combinations given constraints
Precedence Rules: Handling of pre-release versions, build metadata, and special version cases

lib.rs - Application Router and Core Exports

Requirements and Dataflow

Provides centralized module exports and public API surface
Constructs the complete HTTP router with all endpoints and middleware layers
Defines health check endpoints for monitoring and load balancer integration
Serves static content including admin panel interface
Implements comprehensive CORS and security policy configuration

High-level Purpose and Responsibilities

Module Organization: Central module registry and public API definition
Router Assembly: Complete HTTP route construction with nested path organization
Middleware Orchestration: Applies security, authentication, audit, and performance middleware layers
Health Endpoints: Provides deep health checks for database and cache connectivity
Admin Interface: Serves administrative dashboard and management tools
API Versioning: Supports both /api and /api/v1 prefixes for backward compatibility

Key Abstractions and Interfaces

build_router(): Main router construction function accepting shared application state
ready_check(): Database and cache connectivity verification
serve_admin_panel(): Static HTML content serving for administration
Route grouping by functional domain (auth, learners, sessions, analytics, etc.)
Middleware layering with proper ordering for security and functionality

Data Transformations and Flow

Module Exports: Consolidates all subsystem modules into cohesive API
Route Definition: Maps HTTP endpoints to handler functions with proper parameter extraction
Middleware Application: Applies cross-cutting concerns in correct order
State Injection: Provides shared application state to all handlers
Health Verification: Tests database and cache connectivity on demand
CORS Configuration: Environment-specific cross-origin policy application
Response Formatting: Consistent JSON response structure across endpoints

Dependencies and Interactions

Handler Modules: All domain-specific request handlers (auth, learners, sessions, etc.)
Middleware Stack: Security headers, authentication, rate limiting, audit logging
Application State: Database pool, cache connection, configuration access
Monitoring Systems: Health checks, metrics collection, performance tracking
Static Assets: Admin panel HTML and related static resources
WebSocket Support: Real-time session and analytics connections

Architectural Patterns

Layered Middleware: Onion-style middleware application with proper ordering
Route Nesting: Hierarchical path organization by functional domain
Dependency Injection: Shared state container pattern throughout request lifecycle
Health Check Stratification: Multiple levels of health verification (live/ready/detailed)
CORS Policy Management: Environment-specific cross-origin configuration
API Versioning: Multiple endpoint versions for backward compatibility support

config.rs - Environment Configuration Management

Requirements and Dataflow

Loads application configuration from environment variables with fallback defaults
Validates production-specific security requirements and constraints
Manages OAuth provider credentials and TLS certificate configuration
Provides differential privacy parameters and security policy settings
Ensures configuration safety across different deployment environments

High-level Purpose and Responsibilities

Environment Parsing: Converts environment variables to strongly-typed configuration structures
Production Validation: Enforces security policies and prevents weak configurations in production
OAuth Management: Handles Apple and GitHub OAuth provider configuration with validation
TLS Configuration: Manages SSL/TLS settings including Let's Encrypt integration
Security Policies: Defines JWT secrets, CORS policies, rate limiting, and privacy parameters
Development Defaults: Provides sensible defaults for local development environments

Key Abstractions and Interfaces

Config struct: Comprehensive configuration container with all application settings
Environment enum: Deployment environment classification (Development, Staging, Production)
from_env(): Main configuration loading function with environment variable parsing
validate_production_safety(): Production-specific security validation
require_in_production(): Helper for mandatory production configuration values

Data Transformations and Flow

Environment Variables → Typed Config: Parses and validates environment variables
Default Value Application: Applies environment-appropriate defaults where values are missing
OAuth Credential Validation: Verifies OAuth provider configuration completeness
TLS Setting Validation: Ensures proper certificate configuration for HTTPS
Security Policy Enforcement: Validates JWT secrets, CORS origins, and privacy parameters
Production Safety Checks: Prevents deployment with insecure configuration values

Dependencies and Interactions

Environment System: Reads from process environment variables and .env files
OAuth Providers: Validates Apple and GitHub OAuth application credentials
TLS Infrastructure: Coordinates with certificate management and HTTPS setup
Database Layer: Provides database connection URL and pooling configuration
Cache System: Configures Redis/cache connection parameters
Security Middleware: Supplies JWT secrets, CORS policies, and rate limiting settings
Privacy Systems: Provides differential privacy epsilon/delta parameters

Architectural Patterns

Environment-Driven Configuration: Deployment-specific behavior through environment variables
Validation Layers: Multi-stage validation with development vs production requirements
Fail-Fast Strategy: Early configuration validation prevents runtime failures
Security-First Design: Production safety validation prevents insecure deployments
Provider Abstraction: Unified OAuth provider configuration pattern
Privacy by Design: Built-in differential privacy parameter configuration

state.rs - Application State Container

Requirements and Dataflow

Provides centralized dependency container for shared application resources
Manages database connection pool and cache connection lifecycle
Integrates configuration settings and batch job processing services
Ensures consistent resource access patterns across the application
Supports both explicit construction and convenience constructors for different environments

High-level Purpose and Responsibilities

Dependency Injection: Central container for all shared application resources
Resource Management: Coordinates database pools, cache connections, and configuration
Service Integration: Embeds batch job processing and background task services
Compatibility Layer: Maintains backward compatibility with legacy Redis references
Environment Adaptation: Provides different constructors for development vs production setups

Key Abstractions and Interfaces

AppState struct: Primary dependency container with all shared resources
Cloneable design for efficient sharing across async tasks and handlers
Multiple constructors: new() for explicit setup, new_in_memory() for development
Embedded services: BatchJobService integration with shared resource access
Cache compatibility: Dual field names (redis_conn, cache_conn) for migration support

Data Transformations and Flow

Resource Assembly: Combines database pool, cache connection, and configuration
Service Initialization: Creates batch job service with shared resource access
State Cloning: Efficient Arc-based sharing across application components
Legacy Compatibility: Maintains multiple cache connection references during migration
Development Setup: Simplified constructor for local development environments

Dependencies and Interactions

Database Layer: Contains database connection pool for all data access
Cache System: Holds cache connection manager for all caching operations
Configuration: Embedded application configuration for runtime behavior
Batch Services: Integrated batch job processing with shared resource access
All Handlers: Injected into every HTTP handler for resource access
Background Tasks: Shared across monitoring, cleanup, and processing tasks

Architectural Patterns

Dependency Injection Container: Centralized resource management pattern
Resource Sharing: Arc-based efficient resource sharing across async contexts
Service Composition: Embedded service pattern with shared resource access
Migration Support: Dual field names for gradual API migration
Environment Flexibility: Multiple constructors for different deployment contexts
Clone-Friendly Design: Efficient state sharing through reference counting

db.rs - Database Abstraction and Migration Management

Requirements and Dataflow

Provides database-agnostic abstraction layer supporting SQLite and PostgreSQL
Manages connection pooling with configurable parameters and timeouts
Implements comprehensive migration system with rollback capabilities
Handles data type conversions between databases (UUID, JSON storage)
Provides migration validation, backup creation, and history tracking

High-level Purpose and Responsibilities

Database Abstraction: Unified interface across SQLite and PostgreSQL backends
Connection Management: Pool configuration with environment-based parameters
Migration Orchestration: Forward and backward migration execution with safety checks
Data Type Handling: Database-specific UUID and JSON storage abstractions
Migration Safety: Rollback capabilities, validation, and backup creation
Development Support: SQLite for development, PostgreSQL for production flexibility

Key Abstractions and Interfaces

Database struct: High-level database operations wrapper
MigrationManager: Advanced migration handling with rollback support
MigrationInfo: Migration metadata and status tracking
Type aliases (DbPool, DbRow, DbTransaction) for database portability
Data conversion functions (uuid_to_db, json_to_db) for cross-database compatibility

Data Transformations and Flow

Connection Pool Creation: Environment-based pool configuration and initialization
Database-Specific Adaptations: UUID and JSON handling based on backend type
Migration Execution: Forward migration application with dependency tracking
Rollback Processing: Reverse migration execution with transaction safety
Data Type Conversion: Automatic conversion between application and database types
Backup Operations: Database backup creation before major migrations

Dependencies and Interactions

Configuration System: Database URL and pooling parameters from environment
Migration Files: File system integration for migration script loading
Application State: Database pool injection into shared application state
All Handlers: Database access through connection pool abstraction
Monitoring Systems: Connection health checks and pool metrics
Admin Interface: Migration status reporting and management operations

Architectural Patterns

Database Abstraction Layer: Feature flags and type aliases for multi-backend support
Migration as Code: Version-controlled database schema evolution
Rollback Safety: Transactional rollback with integrity checking
Connection Pooling: Resource management with configurable lifecycle policies
Type Safety: Compile-time database feature selection and type checking
Backup Strategy: Automated backup creation for migration safety

cache.rs - In-Memory Redis-Compatible Cache

Requirements and Dataflow

Provides Redis-compatible API for caching operations without external Redis dependency
Implements TTL-based expiration with automatic cleanup mechanisms
Supports common Redis commands (GET, SET, SETEX, DEL, INCR, EXISTS, EXPIRE)
Manages concurrent access through async-safe data structures
Maintains API compatibility with existing Redis client code

High-level Purpose and Responsibilities

Redis API Emulation: Drop-in replacement for Redis client with identical command interface
Memory Management: In-process cache with automatic expiration and cleanup
Concurrency Safety: Thread-safe operations using Arc/Mutex patterns
TTL Management: Automatic expiration handling with background cleanup tasks
Command Translation: Redis command parsing and execution against in-memory storage
Development Simplification: Eliminates Redis server dependency for development/testing

Key Abstractions and Interfaces

ConnectionManager: Redis connection manager compatible interface
InMemoryCache: Core in-memory storage with TTL support
Command: Redis command builder with argument chaining
FromCacheResponse: Type conversion trait for Redis response adaptation
ToArg: Type conversion trait for command arguments
Background cleanup task with automatic expired entry removal

Data Transformations and Flow

Command Construction: Redis-style command building with typed arguments
Argument Serialization: Convert typed arguments to string representations
Command Execution: Parse and execute Redis commands against in-memory store
TTL Processing: Handle expiration times and automatic cleanup
Response Adaptation: Convert internal results to expected Redis response types
Background Cleanup: Periodic removal of expired cache entries

Dependencies and Interactions

Application State: Integrated into shared application state for dependency injection
Handler Layer: Used by all handlers requiring caching functionality
Monitoring Systems: Cache hit/miss metrics and performance tracking
Authentication: Token blacklisting and session management
Rate Limiting: Request counting and window management
Session Storage: Temporary session data and state management

Architectural Patterns

Drop-in Replacement: API-compatible Redis client replacement pattern
Background Processing: Autonomous cleanup task for memory management
Type-Safe Commands: Strongly-typed command arguments and responses
Concurrent Data Structures: Thread-safe access patterns for shared cache
TTL Management: Lazy and proactive expiration handling
Memory Efficiency: Automatic cleanup prevents unbounded memory growth

tls.rs - TLS Certificate Management and HTTPS Server Support

Requirements and Dataflow

Manages TLS certificate lifecycle including acquisition, validation, and renewal
Provides automatic certificate renewal with configurable retry policies and backoff strategies
Supports both Let's Encrypt ACME integration and self-signed certificates for development
Implements comprehensive certificate health monitoring and error tracking
Handles HTTPS server setup with graceful fallback to HTTP-only operation

High-level Purpose and Responsibilities

Certificate Lifecycle: Complete certificate management from acquisition to renewal
Automatic Renewal: Background certificate renewal with configurable policies
Health Monitoring: Certificate expiration tracking and proactive alerting
Development Support: Self-signed certificate generation for local development
Production Integration: ACME/Let's Encrypt integration for automated certificate provisioning
Error Handling: Comprehensive error logging with retry strategies and escalation

Key Abstractions and Interfaces

TlsManager: Primary certificate management service with renewal automation
CertificateStatus: Certificate health and metadata tracking structure
RenewalConfiguration: Configurable policies for automatic certificate renewal
CertificateError: Structured error tracking with severity levels and retry counts
ErrorSeverity: Escalation levels for certificate-related issues (Warning, Error, Critical)

Data Transformations and Flow

Certificate Status Assessment: Regular evaluation of certificate validity and expiration
Renewal Decision Logic: Automated determination of when renewal is required
ACME Integration: Certificate acquisition through Let's Encrypt protocols
Self-Signed Generation: Development certificate creation using rcgen library
TLS Configuration: Rustls server configuration with certificate loading
Error Classification: Structured error tracking with severity-based logging and alerting

Dependencies and Interactions

Configuration System: TLS settings, domain configuration, and certificate paths
File System: Certificate and private key storage and retrieval
ACME Providers: Let's Encrypt integration for certificate provisioning
HTTP Server: ACME challenge handling and certificate validation endpoints
Background Tasks: Automated renewal scheduling and execution
Monitoring Systems: Certificate health metrics and alerting integration
Logging Infrastructure: Structured error tracking and operational visibility

Architectural Patterns

Certificate Lifecycle Management: Complete automation of certificate operations
Retry Strategy Pattern: Configurable backoff and retry policies for reliability
Health Check Integration: Certificate status as part of overall service health
Environment-Specific Behavior: Development self-signed vs production ACME certificates
Error Escalation: Severity-based error handling with appropriate alerting
Background Processing: Autonomous certificate renewal with minimal service impact

error.rs - Centralized Error Management and HTTP Response Mapping

Requirements and Dataflow

Provides unified error handling across all application layers and external dependencies
Maps internal errors to appropriate HTTP status codes and user-friendly messages
Implements structured error logging with unique error IDs for debugging
Handles errors from database, cache, authentication, and core library systems
Supports specialized error types for numerical, statistical, and convergence failures

High-level Purpose and Responsibilities

Error Consolidation: Single error type encompassing all failure modes
HTTP Response Mapping: Automatic conversion from internal errors to HTTP responses
Structured Logging: Detailed error logging with correlation IDs and context
User Experience: User-friendly error messages while preserving debug information
Security: Prevents information leakage through careful error message filtering
Developer Experience: Rich error context for debugging and troubleshooting

Key Abstractions and Interfaces

AppError enum: Comprehensive error type covering all application failure modes
ErrorContext: Internal error logging and ID generation for traceability
AppResult<T>: Type alias providing consistent error handling patterns
IntoResponse implementation: Automatic HTTP response generation from errors
From trait implementations: Seamless conversion from external library errors

Data Transformations and Flow

Error Capture: Conversion from external library errors to unified AppError type
Context Generation: Assignment of unique error IDs and structured logging
Message Filtering: Separation of internal debug information from user messages
HTTP Mapping: Translation of error types to appropriate HTTP status codes
Response Formatting: JSON response construction with consistent structure
Logging Integration: Structured logging with error IDs for correlation

Dependencies and Interactions

All Application Layers: Used throughout handlers, services, and data access layers
External Libraries: Converts errors from SQLx, JWT, UUID, JSON, and core library
HTTP Layer: Integrates with Axum framework for response generation
Monitoring Systems: Provides error metrics and alerting integration points
Logging Infrastructure: Coordinates with tracing system for structured logging
Authentication System: Handles JWT and OAuth-related error conditions

Architectural Patterns

Result Type Propagation: Consistent error handling through Result<T, AppError> pattern
Error Transformation: Systematic conversion from external to internal error types
Logging with Correlation: Unique error IDs for distributed tracing and debugging
Security-Conscious Messaging: Careful separation of internal and external error details
HTTP Response Generation: Automatic mapping from domain errors to HTTP responses
Structured Error Context: Rich error metadata for debugging and monitoring

handlers/mod.rs - HTTP Handler Module Registry

Requirements and Dataflow

Provides centralized module registry for all HTTP request handlers
Organizes handlers by functional domain (auth, analytics, admin, etc.)
Enables consistent import patterns across the application
Supports modular handler organization with clear separation of concerns

High-level Purpose and Responsibilities

Module Organization: Central registry for all HTTP handler modules
Namespace Management: Clear separation of handlers by functional area
Import Coordination: Consistent module exposure for router configuration
Dependency Management: Facilitates handler discovery and organization

Key Abstractions and Interfaces

Module declarations for all handler categories
Consistent naming conventions across handler modules
Clear functional grouping (auth, data, analytics, admin, infrastructure)
Modular organization supporting independent handler development

Data Transformations and Flow

Module Registration: Declaration of all handler modules
Namespace Creation: Functional grouping of related handlers
Import Facilitation: Router configuration and handler discovery
Dependency Organization: Clear handler interdependency structure

Dependencies and Interactions

All Handler Modules: Declares and exposes all individual handler modules
Router Configuration: Used by lib.rs for route-to-handler mapping
Application Architecture: Supports the overall request handling structure
Development Workflow: Enables organized handler development and maintenance

Architectural Patterns

Module Registry Pattern: Centralized handler module organization
Functional Grouping: Handlers organized by business domain
Namespace Separation: Clear boundaries between handler categories
Modular Architecture: Independent handler module development support

handlers/auth.rs - Authentication and User Management

Requirements and Dataflow

Manages complete user authentication lifecycle including registration, login, logout, and token refresh
Implements secure password handling with Argon2 hashing and strength validation
Provides OAuth integration for Apple Sign In and GitHub authentication
Handles session management with JWT tokens and refresh token rotation
Implements account lockout protection and failed login attempt tracking

High-level Purpose and Responsibilities

User Registration: Account creation with comprehensive input validation and security checks
Authentication: Secure login with password verification and account lockout protection
OAuth Integration: Apple and GitHub OAuth flows with state validation and user profile mapping
Token Management: JWT token generation, refresh, and blacklisting for session control
Security Enforcement: Password strength validation, rate limiting, and audit logging
User Profile Management: User information retrieval and account status tracking

Key Abstractions and Interfaces

Registration and login endpoints with comprehensive validation
OAuth authorization URL generation and callback handling
JWT token generation with role-based access control (RBAC)
Session management with token blacklisting and refresh rotation
Password strength validation with common pattern detection
Account lockout mechanisms with configurable retry policies

Data Transformations and Flow

Registration Flow: Input validation → password hashing → user creation → audit logging
Authentication Flow: Credentials validation → lockout checking → token generation → session creation
OAuth Flow: Provider authentication → user profile mapping → account creation/linking → token generation
Token Refresh: Refresh token validation → new access token generation → session continuity
Logout Flow: Token blacklisting → session invalidation → audit logging
Security Monitoring: Failed attempt tracking → lockout enforcement → metrics collection

Dependencies and Interactions

Database Layer: User account storage, OAuth profile management, and session tracking
Cache Layer: Token blacklisting, failed attempt counters, and lockout state management
OAuth Services: Apple and GitHub authentication service integration
Audit System: Comprehensive logging of authentication events and security incidents
Business Metrics: User signup tracking and authentication success/failure metrics
Configuration System: Security policies, OAuth credentials, and lockout parameters
Middleware Integration: JWT claims extraction and role-based authorization

Architectural Patterns

Secure Authentication: Multi-layered security with hashing, validation, and monitoring
OAuth Provider Integration: Standardized OAuth flow handling with state protection
Session Management: JWT-based stateless authentication with refresh token support
Security Monitoring: Comprehensive audit logging and metrics collection
Account Protection: Rate limiting and lockout mechanisms against brute force attacks
User Experience: Seamless authentication flows with proper error messaging

handlers/admin.rs - Administrative Management Interface

Requirements and Dataflow

Provides comprehensive administrative dashboard for system management and monitoring
Manages user accounts, learner profiles, and system configuration
Handles audit trail review and security event monitoring
Supports batch job management and OAuth credential validation
Delivers system health metrics and operational insights

High-level Purpose and Responsibilities

System Administration: User management, configuration updates, and system control
Monitoring Dashboard: Comprehensive system health and performance visibility
Audit Management: Security event review and compliance reporting
Batch Job Control: Background task monitoring and manual job triggering
OAuth Management: Credential validation and authentication system health
Operational Control: System configuration and administrative task execution

Key Abstractions and Interfaces

Administrative dashboard with comprehensive system overview
User and learner management with detailed account information
Audit trail interface with searchable security event logs
Batch job control panel with status monitoring and manual triggers
OAuth validation system with credential health checking

Data Transformations and Flow

Dashboard Generation: System metrics → aggregation → visualization data → administrative interface
User Management: Account queries → user information → management actions → audit logging
Audit Review: Security events → filtering → analysis → compliance reporting
Job Management: Batch job status → control actions → execution monitoring → result reporting
Configuration Management: System settings → validation → updates → change tracking

Dependencies and Interactions

User Management System: Account information and profile management
Audit System: Security event logging and compliance tracking
Batch Job Service: Background task execution and monitoring
OAuth Services: Authentication credential validation and health checking
Monitoring Systems: System health metrics and performance data
Configuration Management: System settings and operational parameters

Architectural Patterns

Administrative Interface: Role-based access control with comprehensive system management
Audit Trail Management: Complete security event tracking with search and reporting
System Monitoring: Real-time operational metrics with alerting and dashboard integration
Batch Processing Control: Administrative oversight of background task execution
Configuration Management: Centralized system setting control with change tracking
Security-First Design: Administrative actions with comprehensive audit logging and access control

handlers/dashboard.rs - System Dashboard and Visualization

Requirements and Dataflow

Provides comprehensive system monitoring dashboard with real-time metrics
Generates visualization-ready data for charts and operational displays
Implements real-time metrics streaming for live dashboard updates
Supports system health monitoring with detailed component status
Integrates OpenTelemetry metrics for observability and monitoring

Key Abstractions and Interfaces

Dashboard data endpoints with aggregated metrics and visualization formats
Real-time metrics streaming with WebSocket or event-driven updates
System health reporting with component status and performance indicators
Database health monitoring with connection pool and query performance metrics
OpenTelemetry integration with distributed tracing and metrics collection

Dependencies and Interactions

Monitoring Systems: System health metrics and performance indicators
Database Layer: Connection pool health and query performance tracking
Real-time Infrastructure: Live metrics streaming and dashboard updates
Observability Stack: OpenTelemetry integration and distributed tracing

handlers/session.rs - Learning Session Management

Requirements and Dataflow

Manages learning session lifecycle including creation, tracking, and completion
Handles task responses and scoring within active learning sessions
Integrates with adaptive learning algorithms for dynamic difficulty adjustment
Provides session replay functionality for analysis and review
Coordinates with learner profiles and topology configurations

High-level Purpose and Responsibilities

Session Lifecycle: Creation, tracking, completion, and archival of learning sessions
Task Management: Integration with core learning algorithms for task generation and adaptation
Response Processing: Recording and scoring of learner responses with feedback generation
Adaptive Learning: Real-time session adjustment based on learner performance
Data Collection: Comprehensive tracking of learning interactions for analytics
Replay System: Session reconstruction for analysis and educational review

Key Abstractions and Interfaces

Session creation with learner validation and topology configuration
Task response submission with scoring and adaptation triggers
Session completion with summary generation and performance metrics
Topology handling for different learning domains (alphabet, music, mathematics)
Integration with core learning algorithms through service layer abstraction

Data Transformations and Flow

Session Creation: Learner validation → topology configuration → session initialization → database persistence
Response Processing: Task response → scoring algorithm → adaptation trigger → database update
Session Progress: Performance tracking → difficulty adjustment → next task generation → learner feedback
Session Completion: Final scoring → summary generation → learner statistics update → session archival
Replay Generation: Session reconstruction → response sequence → performance analytics → visualization data

Dependencies and Interactions

Learner Service: Learner profile management and permission validation
Core Learning Algorithms: Task generation, scoring, and adaptive difficulty adjustment
Database Layer: Session persistence, response tracking, and performance storage
Adaptation Service: Dynamic learning algorithm adjustment based on performance
Business Metrics: Learning effectiveness tracking and engagement analytics
Audit System: Session activity logging and learning interaction tracking

Architectural Patterns

Session State Management: Comprehensive session lifecycle with status tracking
Adaptive Learning Integration: Real-time algorithm adjustment based on performance data
Permission-Based Access: Learner ownership validation and access control
Data Persistence: Detailed session and response tracking for analytics
Service Layer Integration: Clean separation between handlers and core learning logic
Performance Analytics: Comprehensive metrics collection for learning effectiveness analysis

handlers/task.rs - Advanced Task Management

Requirements and Dataflow

Manages complex task generation with advanced learning algorithm integration
Provides sophisticated task configuration with multiple difficulty levels and domains
Implements task scoring and assessment with detailed feedback generation
Supports advanced task types with multi-modal learning and complex interactions
Integrates with adaptive learning systems for dynamic task adjustment

Key Abstractions and Interfaces

Complex task generation with multi-parameter configuration and domain specialization
Advanced scoring algorithms with detailed assessment and feedback generation
Task type management with support for various learning modalities and interaction types
Integration with adaptive learning for real-time task difficulty and type adjustment

Dependencies and Interactions

Core Learning Algorithms: Advanced task generation and adaptive learning integration
Domain Systems: Specialized task types for different learning domains (music, math, language)
Assessment Framework: Sophisticated scoring and feedback generation systems
Adaptive Learning: Dynamic task adjustment based on learner performance and preferences

handlers/experiment.rs - Research Experiment Management

Requirements and Dataflow

Manages research experiment lifecycle including creation, participant enrollment, and results analysis
Provides experiment discovery and joining functionality for research participants
Handles experimental data collection and statistical analysis
Supports data export for research publication and analysis
Implements proper research ethics and participant consent management

High-level Purpose and Responsibilities

Experiment Management: Research study creation, configuration, and lifecycle management
Participant Enrollment: Volunteer recruitment and consent management
Data Collection: Experimental data gathering with proper research protocols
Results Analysis: Statistical analysis and research findings generation
Data Export: Research data extraction with privacy protection and ethics compliance
Ethics Compliance: Research participant protection and consent tracking

Key Abstractions and Interfaces

Experiment creation and configuration with research parameter definition
Participant joining system with consent management and eligibility checking
Results collection and statistical analysis with research methodology compliance
Data export functionality with ethics approval and participant consent verification

Data Transformations and Flow

Experiment Setup: Research design → experiment configuration → participant criteria → ethics approval
Enrollment Process: Participant interest → eligibility verification → consent collection → enrollment confirmation
Data Collection: Participant activities → experimental data capture → protocol compliance → data validation
Results Analysis: Collected data → statistical processing → research findings → publication preparation
Export Processing: Ethics verification → data anonymization → format conversion → secure delivery

Dependencies and Interactions

User System: Research participant management and consent tracking
Learning System: Integration with learning activities for data collection
Statistics Engine: Research-grade statistical analysis and significance testing
Ethics Framework: Research participant protection and consent management
Data Privacy: Participant data protection and anonymization systems
Audit System: Research activity logging and ethics compliance tracking

Architectural Patterns

Research Ethics: Comprehensive participant protection and consent management
Data Collection: Systematic experimental data gathering with protocol compliance
Statistical Analysis: Research-grade analysis with proper methodology and significance testing
Privacy Protection: Participant data anonymization and secure handling
Audit Trail: Complete research activity logging for ethics compliance
Export Control: Secure data export with ethics approval and participant consent verification

handlers/analytics.rs - Learning Analytics and Reporting

Requirements and Dataflow

Provides comprehensive learning analytics including population-wide and individual learner metrics
Generates real-time performance reports and learning effectiveness measurements
Implements adaptive difficulty analysis and learning curve visualization
Supports comparative analysis between different learning strategies and conditions
Delivers live analytics data for dashboard and monitoring systems

High-level Purpose and Responsibilities

Population Analytics: Aggregate learning metrics across all learners and sessions
Individual Performance: Detailed analysis of specific learner progress and effectiveness
Learning Curves: Temporal analysis of skill acquisition and knowledge retention
Strategy Comparison: Comparative effectiveness analysis of different learning approaches
Real-time Monitoring: Live dashboard data for learning system performance
Bottleneck Analysis: Identification of learning challenges and improvement opportunities

Key Abstractions and Interfaces

Population-wide learning metrics with statistical aggregations
Individual learner performance analysis with detailed breakdowns
Adaptive difficulty effectiveness measurement and optimization recommendations
Learning strategy comparison with statistical significance testing
Real-time analytics endpoints for dashboard and monitoring integration

Data Transformations and Flow

Data Aggregation: Raw session data → statistical calculations → performance metrics → analytics reports
Learning Curves: Response history → temporal analysis → skill progression → visualization data
Comparative Analysis: Multiple strategy data → statistical comparison → effectiveness ranking → recommendations
Real-time Processing: Live session data → streaming analytics → dashboard updates → alert generation
Bottleneck Detection: Performance analysis → challenge identification → improvement recommendations → action items

Dependencies and Interactions

Session Data: Learning session records and response histories for analysis
Learner Profiles: Individual learner characteristics and performance baselines
Statistical Engine: Advanced statistical calculations and significance testing
Real-time Systems: Live data streaming and dashboard integration
Business Metrics: Learning effectiveness and engagement measurement
Visualization Systems: Chart generation and dashboard data formatting

Architectural Patterns

Analytics Pipeline: Multi-stage data processing with statistical analysis
Real-time Analytics: Streaming data processing with live dashboard updates
Permission-Based Access: Analytics access control based on user roles and permissions
Statistical Processing: Advanced analytics with confidence intervals and significance testing
Performance Optimization: Efficient query processing for large datasets
Dashboard Integration: Standardized data formats for visualization and reporting systems

models/mod.rs - Data Model Registry and Type Definitions

Requirements and Dataflow

Provides centralized registry for all data model types and structures
Organizes domain models by functional area (users, sessions, experiments, etc.)
Enables consistent data serialization and API response formatting
Supports type safety and validation across the application layer

Key Abstractions and Interfaces

Domain model organization by functional boundaries
Consistent serialization patterns for API responses
Type definitions for database entities and API contracts
Validation and transformation utilities for data models

models/user.rs - User Account and Authentication Models

Requirements and Dataflow

Defines user account data structures with authentication and profile information
Supports multiple authentication providers (local, OAuth, Apple, GitHub)
Handles user registration, login, and profile management data contracts
Provides token response structures for JWT-based authentication

Key Abstractions and Interfaces

User entity with comprehensive profile and authentication data
OAuth integration models for Apple Sign In and GitHub authentication
Authentication request/response structures for login and registration flows
Token management models with access and refresh token handling

models/session.rs - Learning Session Data Models

Requirements and Dataflow

Defines learning session data structures with topology and progress tracking
Supports session lifecycle management from creation to completion
Handles session status tracking and performance summary generation
Provides data contracts for session creation, updates, and analytics

Key Abstractions and Interfaces

Session entity with topology configuration and learning progress
Session status enumeration for lifecycle state management
Session summary structures for performance metrics and analytics
Session creation and update request/response models

models/learner.rs - Learner Profile and Progress Models

Requirements and Dataflow

Defines learner profile data structures with learning preferences and history
Supports learner statistics tracking and performance analytics
Handles learner-session associations and progress monitoring
Provides data contracts for learner management and analytics APIs

Key Abstractions and Interfaces

Learner entity with profile information and learning configuration
Learner statistics models for performance tracking and analytics
Learner creation and update request/response structures
Progress tracking models for learning advancement monitoring

Protocol Models Architecture

Requirements and Dataflow

Core Requirements

Version-controlled research protocol management with semantic versioning
Git-like branching and merging for protocol development
Immutable published protocol versions with deprecation lifecycle
Real-time protocol validation and difference comparison
Federation-aware protocol publishing and synchronization
Performance metrics tracking for protocol effectiveness

Data Flow Patterns

Protocol Creation: CreateProtocolRequest → Validation → Protocol + Initial ProtocolVersion
Version Development: CreateVersionRequest → Validation → ProtocolVersion → Draft Status
Publishing Workflow: PublishVersionRequest → Validation → Status Update → Federation Notification
Branching/Merging: Base Version → Branch Creation → Development → Merge → Version Update
Comparison Analysis: CompareVersionsRequest → Diff Calculation → VersionDifference → Response
Metrics Collection: Protocol Usage → Session Data → Performance Calculation → ProtocolMetrics

High-level Purpose and Responsibilities

Primary Purpose

Provides a comprehensive version control system for research protocols, enabling collaborative development, controlled publication, and performance tracking while maintaining scientific reproducibility and integrity.

Core Responsibilities

Version Management: Semantic versioning with parent-child relationships and branching
Publication Control: Status-based lifecycle from draft through published to deprecated
Collaboration Framework: Git-like branching and merging for team protocol development
Validation System: Rule-based protocol validation with error and warning reporting
Change Tracking: Detailed difference analysis between protocol versions
Performance Analysis: Usage metrics and effectiveness tracking for optimization

Key Abstractions and Interfaces

Core Entities

Protocol: Container entity grouping related protocol versions
ProtocolVersion: Immutable version snapshot with complete definition and metadata
ProtocolBranch: Development branch for experimental protocol variations
ProtocolMetrics: Performance and usage analytics for protocol optimization

Lifecycle Management

ProtocolStatus: Draft, Review, Published, Deprecated, Archived state machine
CreateProtocolRequest/Response: Initial protocol and version creation
PublishVersionRequest: Controlled publication with federation notification
CompareVersionsRequest/Response: Detailed version difference analysis

Validation Framework

ProtocolValidationResult: Comprehensive validation with errors and warnings
ValidationError/Warning: Structured validation feedback with field-level details
VersionDifference: Change detection with typed modification tracking

Data Transformations and Flow

Version Creation Workflow

CreateVersionRequest → Definition Validation → Parent Linking → ProtocolVersion → Draft Status

Publishing Pipeline

Draft Version → Validation → Review Status → Publication → Federation Sync → Current Version Update

Branching and Merging

Base Version → Branch Creation → Development → Comparison → Merge → Version Integration

Difference Analysis

Version A + Version B → Field Comparison → Change Detection → Diff Calculation → Structured Response

Dependencies and Interactions

External Dependencies

chrono: Timestamp management for version tracking and lifecycle events
serde: JSON serialization for protocol definitions and validation rules
sqlx: Database persistence with FromRow mapping and type safety
uuid: Unique identifiers for protocols, versions, and branches

Internal System Interactions

Handlers: Protocol API endpoints consume these models for CRUD and comparison operations
Services: Protocol service layer implements validation, versioning, and federation logic
Federation: Protocol synchronization with partner institutions via federation models
Analytics: Metrics collection integrates with performance tracking systems
Experiments: Protocol definitions link to experiment configurations and execution

Architectural Patterns

Git-Inspired Version Control

Parent-child version relationships with branching support
Semantic versioning with automated progression
Merge tracking between branches and main line
Change history with comprehensive changelog requirements

Immutable Published Versions

Published versions cannot be modified, only deprecated
Definition and validation rules frozen at publication
Timestamped lifecycle events (creation, publication, deprecation)
Hash-based integrity verification for protocol definitions

Status-Based Lifecycle Management

State machine progression: Draft → Review → Published → Deprecated → Archived
Role-based permissions for status transitions
Automated validation requirements for status advancement
Federation notification triggers on publication events

Extensible Validation Framework

JSON-based validation rules with custom logic support
Field-level error and warning reporting
Pluggable validation engine with rule inheritance
Performance impact assessment for validation complexity

Performance-Driven Optimization

Real-time metrics collection from protocol execution
Effectiveness tracking with completion and success rates
Usage pattern analysis for protocol improvement recommendations
A/B testing support through branch comparison metrics

models/experiment.rs - Research Experiment Data Models

Requirements and Dataflow

Defines research experiment data structures with participant management
Supports experimental design configuration and results tracking
Handles participant enrollment and consent management
Provides data contracts for research data collection and analysis

Key Abstractions and Interfaces

Experiment entity with research protocol and participant configuration
Participant enrollment models with consent and eligibility tracking
Experimental results structures for data collection and analysis
Research ethics and compliance data models

Monitoring Module Core Architecture

Requirements and Dataflow

Core Requirements

Unified monitoring infrastructure with histogram-based response time tracking
Thread-safe metrics collection using atomic operations and RwLock patterns
Global metrics collector accessible across the entire application
Request timing utilities with automatic metric recording
Comprehensive endpoint-specific metrics with percentile calculations
System health aggregation with component status tracking

Data Flow Patterns

Request Monitoring: Request Start → Timer Creation → Processing → Timer Completion → Metrics Recording
Histogram Processing: Response Time → Bucket Assignment → Count Increment → Percentile Calculation
Global Aggregation: Individual Metrics → Global Collector → Snapshot Generation → Export
Health Assessment: Component Checks → Status Aggregation → System Health → Dashboard
Performance Analysis: Endpoint Metrics → Percentile Calculation → Trend Analysis → Optimization

High-level Purpose and Responsibilities

Primary Purpose

Serves as the central monitoring infrastructure that orchestrates comprehensive observability across the application, providing both technical performance metrics and business intelligence capabilities.

Core Responsibilities

Metrics Orchestration: Central coordination of all monitoring subsystems
Performance Tracking: Request-level timing with histogram-based percentile calculation
System Health Management: Component health aggregation and status reporting
Data Export: Unified metrics export in multiple formats (Prometheus, JSON)
Business Intelligence: Integration point for business metrics and KPI tracking
Resource Monitoring: Database, connection, and system resource tracking

Key Abstractions and Interfaces

Core Infrastructure

MetricsCollector: Thread-safe global metrics aggregator with atomic counters
ResponseTimeHistogram: Histogram-based response time tracking with percentile calculation
RequestTimer: Automatic request timing utility with endpoint-specific recording
MetricsSnapshot: Point-in-time metrics state for export and analysis

Performance Measurement

EndpointMetrics: Per-endpoint performance tracking with error rates
HistogramBucket: Response time distribution tracking with configurable bounds
PerformanceMetrics: Comprehensive performance analysis with percentile calculations
SlowEndpointInfo: Performance bottleneck identification and analysis

Health Monitoring

SystemHealth: Overall system health with component status aggregation
ComponentStatus: Individual component health with response time tracking
HealthStatus: Standardized health status enumeration (Healthy, Degraded, Unhealthy)
EndpointPerformanceMetrics: Detailed endpoint performance analysis

Data Transformations and Flow

Histogram-Based Response Time Tracking

Response Time → Bucket Selection → Count Increment → Percentile Calculation → Performance Analysis

Global Metrics Aggregation

Individual Operations → Atomic Counter Updates → Metric Collection → Snapshot Generation → Export

Request Lifecycle Tracking

Request Start → Timer Creation → Processing → Completion → Metric Recording → Analysis

Health Status Aggregation

Component Checks → Status Collection → Health Calculation → System Assessment → Dashboard Update

Dependencies and Interactions

External Dependencies

chrono: Timestamp management for metrics and health checks
serde: JSON serialization for metrics export and configuration
std::sync: Atomic operations and thread-safe data structures
tokio::sync: Async-friendly locking primitives for shared state
once_cell: Lazy static initialization for global metrics instance

Internal System Interactions

All Modules: Provides monitoring infrastructure used throughout the application
HTTP Handlers: Request timing and performance measurement integration
Database Layer: Connection and query performance monitoring
Business Logic: KPI tracking and business metrics collection
Health Checks: System health assessment and component monitoring

Architectural Patterns

High-Performance Metrics Collection

Atomic counters for lock-free high-frequency updates
Histogram-based percentile calculation for accurate response time analysis
RwLock optimization for read-heavy endpoint metrics access
Memory-efficient bucket allocation with predefined bounds

Global Singleton Pattern

Lazy-initialized global metrics collector for application-wide access
Thread-safe shared state management with atomic operations
Zero-allocation hot paths for performance-critical metrics recording
Centralized configuration and access control

Extensible Monitoring Framework

Modular design with specialized monitoring components
Plugin-based health check registration system
Configurable histogram buckets for different performance requirements
Standardized interfaces for consistent monitoring patterns

Request Timing Optimization

RAII-based automatic timing with RequestTimer
Minimal overhead measurement techniques
Endpoint-specific metric aggregation with efficient storage
Background processing for complex calculations

Data Export and Integration

Multiple export format support (Prometheus, JSON)
Snapshot-based consistency for metrics export
Integration-friendly APIs for external monitoring systems
Configurable retention and aggregation policies

Health Monitoring Architecture

Requirements and Dataflow

Core Requirements

Comprehensive system health monitoring with multi-component assessment
Real-time health check orchestration with configurable intervals
Deep system monitoring including memory, CPU, and disk utilization
External dependency health verification and circuit breaker integration
Database connection pool monitoring with performance thresholds
Application-level health metrics with business logic validation

Data Flow Patterns

Health Check Cycle: Periodic Timer → Component Checks → Status Aggregation → Health Report
Deep Monitoring: System Resource Collection → Performance Analysis → Threshold Evaluation → Alert Generation
Dependency Verification: External API Calls → Response Time Measurement → Status Determination → Circuit Breaker Update
Database Health: Connection Pool Analysis → Query Performance Check → Resource Utilization → Health Score
Application Health: Business Logic Validation → Feature Availability Check → Service Status → Overall Health

High-level Purpose and Responsibilities

Primary Purpose

Provides a comprehensive health monitoring system that continuously assesses system components, external dependencies, and application-level functionality to ensure optimal service availability and performance.

Core Responsibilities

System Health Assessment: Real-time monitoring of memory, CPU, disk, and network resources
Component Health Verification: Individual component health checks with status aggregation
External Dependency Monitoring: Third-party service availability and performance tracking
Database Health Analysis: Connection pool utilization and query performance monitoring
Application Health Validation: Business logic health checks and feature availability assessment
Alert Management: Threshold-based alerting with severity classification and escalation

Key Abstractions and Interfaces

Core Health Monitoring

EnhancedSystemHealth: Comprehensive system health report with detailed component status
ComponentStatus: Individual component health with response time and error tracking
HealthStatus: Healthy, Degraded, Unhealthy status enumeration with clear semantics
HealthCheckFunction: Pluggable health check interface for extensible monitoring

System Resource Monitoring

DatabasePoolStats: Connection pool utilization with performance metrics
DiskUsageStats: File system monitoring with capacity and utilization tracking
ApplicationHealthMetrics: Application-specific health indicators and thresholds
CircuitBreakerHealth: Circuit breaker status with failure rate and recovery tracking

Alert and Notification

HealthAlert: Health-specific alert with component context and remediation guidance
HealthTrend: Historical health trend analysis with predictive indicators
MaintenanceWindow: Planned maintenance coordination with health check suspension
HealthDashboard: Unified health visualization with status aggregation

Data Transformations and Flow

System Health Collection

Resource Polling → Metric Collection → Threshold Analysis → Status Determination → Health Report

Component Health Aggregation

Individual Checks → Status Collection → Dependency Analysis → Overall Health → Dashboard Update

Alert Generation Process

Health Assessment → Threshold Comparison → Alert Creation → Severity Assignment → Notification Dispatch

External Dependency Monitoring

Service Calls → Response Time Measurement → Availability Check → Circuit Breaker Update → Health Status

Dependencies and Interactions

External Dependencies

axum: HTTP framework integration for health check endpoints
chrono: Timestamp management for health check intervals and trend analysis
serde: JSON serialization for health reports and dashboard integration
std::collections: HashMap for component status tracking and aggregation
tracing: Structured logging integration with health check context

Internal System Interactions

State Management: Integration with application state for component health checks
Database Layer: Connection pool monitoring and query performance assessment
Monitoring System: Integration with global metrics for unified observability
Alert System: Health-based alert generation and notification dispatch
Circuit Breaker: Health status integration with failure detection and recovery

Architectural Patterns

Pluggable Health Check Framework

Extensible health check registration system
Component-specific health check implementations
Async-friendly health check execution with timeout handling
Hierarchical health status aggregation with dependency mapping

Resource Monitoring Integration

System resource monitoring with platform-specific implementations
Memory usage tracking with garbage collection impact assessment
CPU utilization monitoring with load average analysis
Disk space monitoring with threshold-based alerting

Circuit Breaker Integration

Health-aware circuit breaker state management
Failure rate calculation with health impact assessment
Recovery monitoring with health check integration
Graceful degradation based on health status

Dashboard and Alerting

Real-time health dashboard with component status visualization
Configurable alert thresholds with environment-specific tuning
Health trend analysis with predictive failure detection
Maintenance window coordination with health check suspension

Performance Optimization

Efficient health check scheduling with minimal resource impact
Cached health status with configurable TTL for performance
Batch health check execution with parallel processing
Resource usage optimization for continuous monitoring scenarios

Metrics Export Architecture

Requirements and Dataflow

Core Requirements

Multi-format metrics export supporting Prometheus and JSON standards
Configurable metrics endpoint with authentication and access control
Real-time metrics snapshot generation with performance optimization
Prometheus-compatible metric formatting with proper metadata
Content negotiation for different metric export formats
Metrics availability control with feature flag integration

Data Flow Patterns

Metrics Collection: Global Metrics → Snapshot Generation → Format Conversion → HTTP Response
Prometheus Export: Metrics Request → Authentication Check → Snapshot Retrieval → Prometheus Format → Response
JSON Export: API Request → Permission Validation → Metrics Aggregation → JSON Serialization → Response
Format Negotiation: Accept Headers → Format Selection → Appropriate Serialization → Content-Type Response
Access Control: Request → Authentication → Authorization → Metrics Filtering → Secure Response

High-level Purpose and Responsibilities

Primary Purpose

Provides standardized metrics export capabilities that enable integration with monitoring systems like Prometheus, Grafana, and custom dashboards while ensuring secure access and proper data formatting.

Core Responsibilities

Multi-Format Export: Support for Prometheus text format and JSON APIs
Secure Access: Authentication and authorization for metrics endpoints
Performance Optimization: Efficient snapshot generation and serialization
Standard Compliance: Prometheus metric format specification adherence
Content Negotiation: Automatic format selection based on client preferences
Configuration Management: Feature flag control and endpoint customization

Key Abstractions and Interfaces

Export Interfaces

prometheus_metrics(): Prometheus text format export with standard metadata
json_metrics(): JSON format export with structured metric data
format_prometheus_metrics(): Prometheus format conversion with proper naming
MetricsSnapshot: Unified metrics data structure for format conversion

Access Control

Authentication Check: Configurable authentication requirements for metrics access
Feature Flag Integration: Metrics availability control via configuration
Permission Validation: Role-based access control for sensitive metrics
Rate Limiting: Request throttling for metrics endpoints

Format Conversion

PrometheusFormatter: Text format conversion with proper escaping and metadata
JSONSerializer: Structured JSON output with hierarchical organization
ContentTypeNegotiation: Automatic format selection based on Accept headers
MetricNormalization: Consistent naming and value formatting across formats

Data Transformations and Flow

Prometheus Export Pipeline

Global Metrics → Snapshot Generation → Prometheus Format Conversion → Text Response → Client

JSON Export Pipeline

Metrics Collection → Data Aggregation → JSON Serialization → Structured Response → Dashboard

Authentication Flow

HTTP Request → Configuration Check → Authentication Validation → Metrics Access → Response Generation

Format Selection Process

Client Request → Accept Header Analysis → Format Determination → Appropriate Serialization → Response

Dependencies and Interactions

External Dependencies

axum: HTTP framework integration for endpoint handling and response generation
serde: JSON serialization for structured metric export
std::sync: Thread-safe access to shared metrics data
HeaderMap: HTTP header processing for content negotiation and authentication

Internal System Interactions

Global Metrics: Direct integration with centralized metrics collection system
Configuration: Feature flag and security configuration integration
Authentication: User authentication and authorization system integration
State Management: Application state access for configuration and security context

Architectural Patterns

Multi-Format Support

Pluggable format conversion with extensible serialization strategies
Content-type negotiation with automatic format selection
Standardized internal representation with format-specific conversion
Consistent metric naming and value representation across formats

Secure Metrics Access

Configuration-driven authentication requirements
Feature flag integration for metrics endpoint availability
Role-based access control with granular permission management
Rate limiting and abuse prevention for metrics endpoints

Performance Optimization

Cached snapshot generation with configurable refresh intervals
Efficient serialization strategies for different output formats
Memory-conscious large dataset handling with streaming support
Background metrics aggregation to minimize request latency

Standards Compliance

Prometheus exposition format compliance with proper metadata
OpenMetrics standard support for enhanced interoperability
JSON-based metrics API following RESTful conventions
Proper HTTP status codes and error handling for all scenarios

Extensibility Framework

Plugin architecture for custom metric formats
Configurable metric filtering and transformation
Custom authentication provider integration
Extensible content negotiation with new format support

Performance Analytics Architecture

Requirements and Dataflow

Core Requirements

Detailed endpoint performance analysis with percentile calculations
Real-time performance metric aggregation and trend analysis
Slow endpoint identification with optimization recommendations
Configurable performance monitoring with feature flag control
Comprehensive performance dashboards with historical data
Performance regression detection and alerting capabilities

Data Flow Patterns

Performance Collection: Endpoint Metrics → Percentile Calculation → Performance Analysis → Dashboard Update
Trend Analysis: Historical Data → Performance Comparison → Trend Identification → Alert Generation
Optimization Insights: Slow Query Detection → Bottleneck Analysis → Recommendation Generation → Action Items
Real-time Monitoring: Live Metrics → Performance Assessment → Threshold Evaluation → Alert Dispatch
Dashboard Generation: Metric Aggregation → Performance Summary → Visualization Data → Client Response

High-level Purpose and Responsibilities

Primary Purpose

Provides comprehensive performance analytics and monitoring capabilities that enable proactive identification of performance bottlenecks, optimization opportunities, and system health assessment through detailed metric analysis.

Core Responsibilities

Performance Analysis: Detailed endpoint performance assessment with statistical analysis
Percentile Calculation: Accurate response time percentiles using histogram-based methods
Bottleneck Identification: Slow endpoint detection with performance impact analysis
Trend Monitoring: Historical performance tracking with regression detection
Optimization Guidance: Performance improvement recommendations based on metric analysis
Dashboard Integration: Real-time performance visualization and reporting

Key Abstractions and Interfaces

Performance Analysis

get_endpoint_performance(): Comprehensive endpoint performance analysis with percentiles
EndpointPerformanceDetails: Detailed performance metrics including P50, P95, P99, P999
EndpointPerformanceMetrics: Aggregated performance data for all monitored endpoints
SlowEndpointInfo: Performance bottleneck identification with optimization context

Metric Calculation

PercentileCalculator: Histogram-based percentile calculation for accurate analysis
ErrorRateAnalysis: Error rate calculation and trend analysis
ResponseTimeAnalysis: Comprehensive response time statistical analysis
ThroughputCalculation: Request rate and capacity analysis

Performance Dashboards

PerformanceDashboard: Real-time performance overview with key metrics
PerformanceTrend: Historical performance tracking with change analysis
OptimizationReport: Performance improvement recommendations and action items
PerformanceAlert: Threshold-based performance degradation alerts

Data Transformations and Flow

Performance Metric Calculation

Raw Endpoint Metrics → Histogram Analysis → Percentile Calculation → Performance Summary → Dashboard

Trend Analysis Pipeline

Historical Data → Performance Comparison → Trend Detection → Regression Analysis → Alert Generation

Optimization Analysis Flow

Performance Metrics → Bottleneck Detection → Impact Analysis → Recommendation Generation → Action Items

Real-time Monitoring Process

Live Metrics → Performance Assessment → Threshold Comparison → Alert Evaluation → Notification Dispatch

Dependencies and Interactions

External Dependencies

axum: HTTP framework integration for performance monitoring endpoints
std::collections: HashMap for efficient endpoint metric storage and retrieval
std::time: Duration measurement for performance timing and analysis
tracing: Structured logging for performance monitoring and debugging

Internal System Interactions

Global Metrics: Direct integration with centralized metrics collection system
Configuration: Feature flag control for performance monitoring availability
State Management: Application state access for configuration and context
Alert System: Performance-based alert generation and notification integration

Architectural Patterns

Statistical Performance Analysis

Histogram-based percentile calculation for accurate response time analysis
Error rate calculation with statistical significance testing
Throughput analysis with capacity planning insights
Performance regression detection with automated alerting

Real-time Performance Monitoring

Live metric collection with minimal performance overhead
Configurable monitoring intervals for different performance requirements
Automatic performance threshold detection with adaptive baselines
Real-time dashboard updates with efficient data aggregation

Performance Optimization Framework

Automated slow endpoint detection with configurable thresholds
Performance bottleneck analysis with root cause identification
Optimization recommendation engine based on performance patterns
A/B testing support for performance improvement validation

Dashboard and Visualization

Comprehensive performance dashboards with drill-down capabilities
Historical performance trend visualization with comparative analysis
Real-time performance monitoring with customizable alerts
Performance report generation with actionable insights

Scalable Monitoring Architecture

Efficient metric aggregation with minimal memory footprint
Background performance analysis to avoid impacting request handling
Configurable monitoring scope with endpoint-specific control
Integration-friendly APIs for external performance monitoring tools

OpenTelemetry Integration Architecture

Requirements and Dataflow

Core Requirements

OpenTelemetry-compatible metrics export with standard format compliance
Distributed tracing integration with trace and span management
Service identification with environment and version tracking
Custom attribute support for application-specific context
Resource metadata integration for service discovery and monitoring
Standards-compliant metric types and measurement units

Data Flow Patterns

Metrics Export: Internal Metrics → OTel Format Conversion → Export Pipeline → Collector
Trace Generation: Request Start → Span Creation → Context Propagation → Span Completion → Export
Resource Attribution: Service Metadata → Resource Annotation → Metric Attribution → Context Enhancement
Custom Attributes: Application Context → Attribute Extraction → Metric Enrichment → Export
Distributed Context: Trace Context → Span Relationships → Cross-Service Correlation → Analysis

High-level Purpose and Responsibilities

Primary Purpose

Provides OpenTelemetry-compliant observability integration that enables standardized metrics export, distributed tracing, and seamless integration with modern observability platforms and tools.

Core Responsibilities

Standards Compliance: OpenTelemetry specification adherence for interoperability
Metrics Export: Convert internal metrics to OTel format with proper metadata
Distributed Tracing: Span lifecycle management with context propagation
Resource Management: Service identification and environment context
Custom Attribution: Application-specific context and metadata enrichment
Integration Support: Seamless integration with OTel collectors and platforms

Key Abstractions and Interfaces

Core OTel Integration

OtelMetricsExporter: Service-aware metrics exporter with custom configuration
OtelMetric: Standard-compliant metric representation with resource attribution
OtelResource: Service identification and deployment context information
OtelSpan: Distributed tracing span with parent-child relationship tracking

Metric Standards

MetricType: Counter, Gauge, Histogram, Summary classification system
CustomAttributes: Application-specific context and metadata support
ResourceAttributes: Service, version, environment, and instance identification
TimestampManagement: UTC timestamp handling for metric correlation

Tracing Integration

TraceContext: Distributed trace correlation with trace and span IDs
SpanRelationships: Parent-child span hierarchy for request flow tracking
OperationNaming: Consistent operation identification for trace analysis
ContextPropagation: Cross-service trace context transmission

Data Transformations and Flow

Metrics Transformation Pipeline

Internal Metrics → OTel Format Conversion → Resource Attribution → Export Serialization → Collector

Trace Context Flow

Request → Span Creation → Context Injection → Service Calls → Context Extraction → Span Completion

Resource Attribution Process

Service Configuration → Resource Metadata → Metric Attribution → Context Enrichment → Export

Custom Attribute Processing

Application Context → Attribute Extraction → Validation → Metric Enhancement → Export Pipeline

Dependencies and Interactions

External Dependencies

chrono: UTC timestamp management for metric correlation and trace timing
serde: JSON serialization for OTel format compliance and export
std::collections: HashMap for custom attributes and resource metadata
tracing: Integration with Rust tracing ecosystem for span management

Internal System Interactions

Global Metrics: Integration with internal metrics collection for export conversion
HTTP Layer: Request tracing with automatic span generation and context propagation
Configuration: Service metadata and OTel exporter configuration
External Systems: OTel collector integration for metrics and trace export

Architectural Patterns

Standards-Compliant Export

OpenTelemetry specification adherence for maximum interoperability
Standard metric types with proper unit and description metadata
Resource attribution following OTel semantic conventions
Compatible serialization formats for collector integration

Distributed Tracing Integration

Automatic span creation with request-response lifecycle tracking
Context propagation using standard OTel headers and formats
Parent-child span relationships for complex request flows
Integration with external tracing systems and APM platforms

Service Discovery and Identification

Comprehensive resource metadata for service identification
Environment-aware configuration for deployment-specific context
Version tracking for release correlation and analysis
Instance identification for horizontal scaling scenarios

Custom Context Enhancement

Flexible attribute system for application-specific metadata
Configurable context extraction from request and application state
Custom resource attributes for enhanced service identification
Integration with business logic for domain-specific context

Performance and Scalability

Efficient metric conversion with minimal overhead
Batch export capabilities for high-throughput scenarios
Configurable export intervals and buffer management
Resource-conscious span creation and management

Xilem App Abstraction Documentation

This directory contains high-level abstraction documentation for the xilem-app source code, organized to mirror the original source structure.

File Types

*.rs.abs.md - Abstraction summaries for each Rust source file
- Conceptual overview of what the component does
- Key data flows and state management patterns
- Main responsibilities and interactions
- Dependencies on other components
- User-facing functionality
IMPLICATIONS.md - Directory-level dependency documentation
- Backend (web-backend) requirements
- Core library (abcdeez_core) requirements
- API endpoints needed
- Data structures and types required
- Business logic dependencies

Directory Structure

src/
├── app.rs.abs.md              # Main application routing
├── lib.rs.abs.md              # Library exports
├── main.rs.abs.md             # Entry point
├── IMPLICATIONS.md            # Root-level dependencies
│
├── components/                # Reusable UI components
├── models/                    # Data models and structures
├── services/                  # Business logic services
├── state/                     # Application state management
├── utils/                     # Utility functions
└── views/                     # UI view components

Purpose

This documentation provides:

High-level understanding of the application architecture
Dependency mapping between UI, backend, and core library
Requirements analysis for backend API development
Integration points with the core learning system
Conceptual overview for developers and architects

Key Insights

The xilem-app is a sophisticated adaptive learning platform with:

Offline-first architecture with SQLite storage and synchronization
Real-time learning sessions via WebSocket communication
Advanced interaction tracking for learning analytics
Gamification system for user engagement
Multiple task types with rich visualizations
Clean separation between UI, services, and core logic

Generated on: $(date)

app.rs - Main Application Logic Router

Conceptual Overview

Central routing logic that determines which UI screen to display based on application state. Acts as the main dispatcher for different application views and handles loading states.

Key Data Flows

Receives AppState mutations from user interactions
Routes to appropriate view modules based on current_screen
Applies global theming to the root container
Manages loading screen display

Main Responsibilities

Screen navigation and routing logic
Global theme application
Loading state management
Root container styling and layout

Dependencies on Other Components

state::AppState - Global state for screen routing
views::* - All view modules for different screens
Theme system for consistent styling

User-Facing Functionality

Seamless navigation between different app screens
Consistent theming across all views
Loading indicators during state transitions

main.rs - Application Entry Point

Conceptual Overview

Main entry point for the ABCDEEZ Xilem-based learning application. Responsible for bootstrapping the entire UI framework and initializing the application lifecycle.

Key Data Flows

Creates initial AppState with default values
Initializes tracing for debugging/logging
Sets up Xilem application with window configuration
Delegates main application logic to app::app_logic
Handles the main event loop

Main Responsibilities

Application initialization and startup
Window configuration (size, title, minimum dimensions)
Tracing/logging setup
Event loop management
Application termination handling

Dependencies on Other Components

app::app_logic - Main UI logic routing
state::AppState - Global application state management
Xilem framework for UI runtime

User-Facing Functionality

Creates the main application window
Sets up the learning application interface
Handles application lifecycle (start/stop)

components/mod.rs - Reusable UI Components Module

Conceptual Overview

Placeholder module for reusable UI components that can be shared across different views. Currently contains TODO items for planned components like charts, timers, progress indicators, and navigation elements.

Key Data Flows

Will provide reusable UI patterns across the application
Planned to support data visualization and interaction components
Will enable consistent UI patterns and reduce code duplication

Main Responsibilities

Reusable component organization (planned)
Common UI pattern definitions (future)
Cross-view component sharing (not yet implemented)

Dependencies on Other Components

Will depend on Xilem UI framework
Will integrate with theme system
Will support various data models

User-Facing Functionality

Planned: Data visualization components (charts)
Planned: Interactive timers and progress indicators
Planned: Navigation and card container components
Currently: Placeholder for future development

services/mod.rs - Business Logic Services Module

Conceptual Overview

Central hub for all business logic services that handle backend communication, data persistence, timing, learning algorithms, and interaction tracking. Provides clean interfaces between UI and business logic.

Key Data Flows

Coordinates between UI layer and backend services
Manages data persistence and synchronization
Handles real-time communication via WebSocket
Processes learning algorithms and adaptive feedback
Tracks user interactions for analytics

Main Responsibilities

Service layer organization and exports
Business logic separation from UI concerns
Backend integration interfaces
Data synchronization coordination
Learning algorithm integration

Dependencies on Other Components

All service submodules (api, websocket, sync, storage, timing, adaptive_learning, interaction_tracking)

User-Facing Functionality

Seamless backend integration
Offline capability with synchronization
Adaptive learning experiences
Detailed interaction analytics
Real-time collaborative features

services/api.rs - HTTP API Client Service

Conceptual Overview

Comprehensive HTTP client for communication with the web-backend. Handles authentication, session management, task generation, learner analytics, synchronization, and gamification features.

Key Data Flows

Manages authentication tokens and refresh cycles
Sends/receives session data and task responses
Synchronizes offline data with backend
Retrieves gamification and leaderboard data
Handles error responses and timeouts

Main Responsibilities

HTTP client configuration and management
Authentication and token management
Session lifecycle API calls
Task generation and hint requests
Response submission and feedback retrieval
Learner profile and statistics management
Offline data synchronization
Gamification API integration

Dependencies on Other Components

models::* - Data models for API communication
Reqwest for HTTP client functionality
Serde for JSON serialization/deserialization

User-Facing Functionality

Seamless authentication experience
Real-time task generation and feedback
Learner progress synchronization
Gamification features (levels, achievements, leaderboards)
Offline learning with eventual consistency

services/adaptive_learning.rs - Adaptive Learning Service Integration

Conceptual Overview

Integration layer with abcdeez_core adaptive learning systems. Manages learner models, task selection, response processing, and provides custom intervention logic for the UI application.

Key Data Flows

Integrates with abcdeez_core AdaptiveScheduler and LearnerModel
Processes task responses for model updates
Selects next tasks based on learning progression
Provides custom hint and intervention systems
Exports learner models and Bayesian models

Main Responsibilities

abcdeez_core integration and orchestration
Adaptive task selection algorithms
Response processing and model updates
Custom intervention and hint generation
Learner metrics calculation and reporting
Model export for persistence and analysis

Dependencies on Other Components

abcdeez_core::* - Core learning algorithms and models
models::* - Application data models
Integration between core algorithms and UI needs

User-Facing Functionality

Personalized learning experiences
Adaptive difficulty progression
Intelligent hint systems
Learning progress optimization
Performance-based task selection

services/websocket.rs - Real-time WebSocket Communication

Conceptual Overview

WebSocket client for real-time communication with the backend during learning sessions. Provides bi-directional messaging for live task delivery, response submission, hints, interventions, and performance updates.

Key Data Flows

Establishes authenticated WebSocket connections
Handles real-time task delivery and response submission
Processes live feedback and interventions
Manages heartbeat and connection health
Supports session pause/resume functionality

Main Responsibilities

WebSocket connection management
Authentication and session binding
Real-time message handling (client/server)
Connection health monitoring and reconnection
Live task interaction support
Performance and statistics updates

Dependencies on Other Components

models::Task - Task data structures
Tokio for async WebSocket operations
Serde for message serialization

User-Facing Functionality

Real-time learning experiences
Immediate feedback on responses
Live adaptive interventions
Session progress updates
Seamless connection management with auto-reconnect

services/storage.rs - Local SQLite Storage Service

Conceptual Overview

Local data persistence service using SQLite for offline storage of user data, sessions, responses, learner models, and cache. Provides comprehensive data management with support for synchronization and cleanup operations.

Key Data Flows

Persists user profiles and session data locally
Manages pending response queue for offline sync
Caches learner models and task data
Stores gamification data and achievements
Supports generic key-value storage for extensibility

Main Responsibilities

SQLite database schema management
User and session data persistence
Offline response queue management
Learner model caching and retrieval
Task caching with TTL support
Gamification data storage
Generic key-value storage operations
Database cleanup and maintenance

Dependencies on Other Components

models::* - Data models for storage
abcdeez_core::learning::learner::LearnerModel - Learning model integration
Rusqlite for SQLite database operations
Serde for JSON serialization in storage

User-Facing Functionality

Offline application functionality
Fast data access and caching
Data persistence across app sessions
Efficient storage management
Background data cleanup

services/sync.rs - Offline Data Synchronization Service

Conceptual Overview

Manages offline data synchronization with conflict resolution, batch processing, and background sync operations. Ensures data consistency between local storage and backend systems.

Key Data Flows

Queues responses for offline synchronization
Performs batch uploads with error handling
Handles remote updates and conflict resolution
Manages persistent storage integration
Supports background synchronization scheduling

Main Responsibilities

Offline response queue management
Batch synchronization with configurable sizes
Conflict resolution for concurrent updates
Background sync task coordination
Storage integration for crash recovery
Sync status and metrics tracking

Dependencies on Other Components

models::* - Response and sync data models
services::api::ApiClient - Backend communication
services::storage::StorageService - Local persistence
Tokio for async operations and timers

User-Facing Functionality

Seamless offline learning capability
Automatic background data synchronization
Data consistency across devices
Recovery from network interruptions
Transparent sync status indicators

views/mod.rs - UI View Components Module

Conceptual Overview

Central organization point for all user interface view components. Exposes different screen implementations and specialized visual components for the Xilem-based UI system.

Key Data Flows

Organizes UI components by functional area
Provides access to specialized task visual components
Enables modular UI development and testing

Main Responsibilities

UI component organization and exports
View module coordination
Interface definition for different app screens
Specialized component access (task visuals)

Dependencies on Other Components

All view submodules (auth, dashboard, learning, settings, task visuals)

User-Facing Functionality

Comprehensive user interface across all app functions
Consistent visual design and interaction patterns
Modular and maintainable UI architecture

views/auth.rs - Authentication User Interface

Conceptual Overview

Complete authentication interface supporting login and signup flows with OAuth integration. Provides form handling, validation, loading states, and seamless navigation between authentication modes.

Key Data Flows

Manages authentication form state (username, password, email)
Handles authentication loading states and feedback
Integrates with OAuth providers (Apple, GitHub)
Updates global user state upon successful authentication
Navigates between login/signup screens

Main Responsibilities

Login and signup form interfaces
Authentication state management
OAuth integration UI (Apple, GitHub)
Form validation and error display
Loading state indication
Navigation between auth screens

Dependencies on Other Components

state::AppState - Global state for user authentication
state::UserState - User profile creation
Theme system for consistent styling

User-Facing Functionality

User account creation and login
OAuth sign-in options (Apple, GitHub)
Form validation and error feedback
Loading indicators during authentication
Seamless transition to main application

views/dashboard.rs - Main Dashboard Interface

Conceptual Overview

Primary landing screen after authentication featuring welcome messages, learning statistics, quick actions, and recent activity. Serves as the central hub for navigation to all app features.

Key Data Flows

Displays user progress and learning statistics
Provides quick navigation to major app functions
Shows recent learning activity and achievements
Integrates with bottom navigation system

Main Responsibilities

Welcome and overview interface
Learning statistics display (sessions, accuracy, streaks)
Quick action navigation (practice, progress, leaderboard)
Recent activity timeline
Bottom navigation integration
Header with settings access

Dependencies on Other Components

state::AppState - Global state for user and navigation
Theme system for consistent styling
Navigation integration with all major screens

User-Facing Functionality

Personalized welcome experience
Quick access to learning sessions
Progress overview and motivation
Easy navigation to all app features
Recent activity and achievement highlights

views/core_task_visual.rs - Core Task Visual Representations

Conceptual Overview

Specialized visual components for core learning tasks from abcdeez_core. Provides rich, interactive visual presentations for different core task types including ordering, succession, jumps, segments, and topological operations.

Key Data Flows

Receives core task types and renders appropriate visuals
Uses theme colors for consistent visual presentation
Creates interactive visual representations of abstract learning concepts
Supports various core task categories with distinct visual patterns

Main Responsibilities

Visual rendering for core task types (PairwiseOrder, Successor, KJump, etc.)
Consistent visual design across all core task presentations
Theme integration for visual consistency
Interactive visual elements that enhance learning comprehension
Complex task visualization (topological sorts, minimal/maximal elements)

Dependencies on Other Components

abcdeez_core::tasks::core::TaskType - Core task definitions
state::AppState - For theme access
Xilem UI framework for visual components

User-Facing Functionality

Rich visual learning experiences for core concepts
Intuitive visual representations of abstract relationships
Consistent visual language across different task types
Enhanced comprehension through visual learning
Interactive elements that support task understanding

views/extended_task_visual.rs - Extended Task Visual Representations

Conceptual Overview

Advanced visual components for extended learning tasks that go beyond core concepts. Provides sophisticated visual presentations for complex tasks including boundary bridging, reverse operations, directional comparisons, and transfer learning scenarios.

Key Data Flows

Renders extended task types with specialized visual patterns
Supports complex multi-step task visualizations
Integrates theme colors for visual consistency
Handles diverse extended task categories with unique visual approaches

Main Responsibilities

Visual rendering for extended task types (BetweenQuery, BoundaryBridging, etc.)
Complex multi-element task presentations
Advanced learning concept visualization
Theme integration for consistent appearance
Support for transfer learning and projection switching visuals

Dependencies on Other Components

abcdeez_core::tasks::extended::ExtendedTaskType - Extended task definitions
state::AppState - Theme access for styling
Xilem UI framework for advanced visual components

User-Facing Functionality

Sophisticated visual learning experiences for advanced concepts
Multi-step task visualizations with clear progression indicators
Complex relationship visualization (boundaries, projections, transfers)
Enhanced understanding through advanced visual metaphors
Interactive elements for complex learning scenarios

views/learning.rs - Interactive Learning Interface

Conceptual Overview

Core learning experience interface that presents adaptive tasks, manages user responses, tracks performance, and provides feedback. Supports multiple task types with specialized visual presentations and real-time progress tracking.

Key Data Flows

Displays current learning tasks with visual presentations
Captures user responses and timing data
Updates struggle indicators and performance metrics
Manages hint system and task progression
Integrates with adaptive learning algorithms

Main Responsibilities

Task presentation with specialized visuals
Response capture and submission
Real-time performance tracking (accuracy, timing)
Progress indication and session management
Hint system integration
Loading states for task transitions
Session creation and management

Dependencies on Other Components

models::Task - Task data structures
state::SessionState - Learning session management
views::core_task_visual - Core task visualizations
views::extended_task_visual - Extended task visualizations
Integration with abcdeez_core for task generation

User-Facing Functionality

Interactive learning task experiences
Multiple task types (core, extended, music, navigation, boundary)
Real-time feedback and performance tracking
Adaptive hint system
Progress visualization during sessions
Seamless task transitions and loading

views/settings.rs - Settings and Profile Management Interface

Conceptual Overview

Comprehensive settings interface for user profile management, preferences, theme selection, and account operations. Provides organized sections for different types of settings with intuitive controls.

Key Data Flows

Manages user profile display and editing
Handles theme preference changes
Processes notification and sound settings
Manages account operations (export, privacy, sign out)

Main Responsibilities

User profile section with avatar and info display
Theme selection (Light/Dark/Auto) with live preview
Preference management (notifications, sounds)
Account management (data export, privacy, deletion)
Settings organization into logical sections
Navigation back to previous screens

Dependencies on Other Components

state::AppState - Global state for settings
state::ThemeMode - Theme management
state::UserState - User profile data
Theme system for consistent styling

User-Facing Functionality

User profile viewing and editing
Theme customization with immediate preview
Notification and sound preference control
Account data management and export
Privacy settings access
Account deletion and sign out options