SEC 01
Pipeline Architecture
20-STEP PIPELINECollectors
29
PARALLEL
29 Data Collectors
Parallel ingestion via asyncio.gather across 6 intelligence modules. RSS, OSINT, financial, geopolitical, satellite, and social feeds.
Avg cycle
~48s
Convergence Engine
v2
D-S FUSION
Convergence Engine v2
Dempster-Shafer evidence fusion, entity-pair clustering, trigram dedup, geographic convergence on 1 deg spatial grid. Produces 40-45 clusters per run.
Processing
~3s
Processors
51
MODULES
51 Processing Modules
Anomaly detection, geo enrichment, entity graph, weak signal detection, instability scoring, prediction divergence, market-news divergence, and more.
Total
~18s
Signal Classifier
T1-T3
16 CATEGORIES
Tiered Signal Classification
T1 (Revenue): Brazil macro, whale signals. T2 (Intel): Geopolitical conflicts, sanctions, markets. T3 (Background): Cyber, climate, AI. Source quality scoring 1-4.
Per cycle
10-13 signals
ML Models
3
ACTIVE
Machine Learning Scoring
Structural Risk Filter (RF, AUC=0.94), Event Accelerator (GBM, R2=0.77), LLM Scorer (Claude Haiku + Gemini Flash fallback). Country-level + event-level risk.
AUC best
0.94
Trade Execution
SHADOW
KELLY SIZING
Execution Engine
Kelly Criterion sizing, Bayesian win-rate estimation, E12 strategy (Oracle+NO+NoSports), TP/SL/trailing exits, $100/day cap. Currently in SHADOW mode ($0.01 real orders).
Strategy
E12 (96.8% WR)
CYCLE TIME
74
sec
Collection (65%)
Processing (25%)
Output (10%)
SEC 02
Data Sources
29 COLLECTORS · 6 MODULES
RTM
Real-Time Monitor
8 collectors
RSS Feeds
Open-source intelligence
GDELT
Global event detection
Bluesky
Social OSINT
OpenSky / ADS-B
Military flight tracking
Telegram Inbound
15 OSINT channels
Brave Search
Web + news intelligence
GNews
80K+ news sources
Exa.ai
Neural semantic search
GR
Geopolitical Risk
6 collectors
ACLED
Armed conflict events
GDELT Stability
Country stability scores
V-Dem
Democracy indices
REIGN
Leader & regime data
VIEWS Forecast
Conflict predictions
GPR Index
Geopolitical risk scores
FFT
Financial Flow Tracking
11 collectors
BCB
BRL/Selic macro data
Polymarket
Prediction market events
PM Whales
Insider flow detection
Metaculus
Community forecasts
Kalshi
Regulated PM (US)
Forex
12 currency pairs
TankerTrackers
Sanctioned vessel tracking
Manifold Markets
4-way PM divergence
World Bank
Macro indicators
CoinGecko
Crypto risk proxy
Alpha Vantage
Commodities + sentiment
CT
Cyber & Threat
4 collectors
Nuclear Seismic
USGS earthquake/nuclear
FIRMS
Satellite fire detection
IODA
Internet outage detection
AIS Stream
Maritime vessel tracking
SEC 03
Processing Modules
CORE INTELLIGENCEConvergence Engine
DEMPSTER-SHAFER + ENTITY CLUSTERING
Evidence fusion across multiple intelligence sources. Entity-pair co-occurrence clustering with trigram dedup. Geographic convergence on 1-degree spatial grid with 6 event categories.
40-45
CLUSTERS / CYCLE
~3s
PROCESSING
Anomaly Detector
WELFORD ONLINE + CSD + MULTI-TIER
O(1) streaming mean/variance computation. Three severity tiers: elevated (z>=1.5), high (z>=2.0), critical (z>=3.0). Auto-migration from legacy array baselines. Change-point detection for regime shifts.
O(1)
COMPLEXITY
3
SEVERITY TIERS
Signal Classifier
T1 REVENUE / T2 INTEL / T3 BACKGROUND
16 category taxonomy. Source quality scoring (SQ 1-4). Domain-based trust calibration: Bellingcat/ACLED=4, Reuters/BBC=2. Entity name cleaning, proper noun blocklist filtering.
16
CATEGORIES
10-13
SIGNALS / CYCLE
Structural Risk Filter
LOGISTIC REG + RANDOM FOREST (LOOCV)
ML-backed country instability prediction. 12 features from REIGN, V-Dem, World Bank, neighborhood contagion, and regime interaction terms. Leave-one-out cross-validation.
0.94
AUC (RF)
0.12
BRIER SCORE
Event Accelerator
GRADIENT BOOSTING REGRESSION
Daily event-driven instability scoring. 9 features from FIRMS fires, GDELT events, RSS articles, Telegram messages, and IODA outages. Event acceleration = event_score - structural_baseline.
0.77
R-SQUARED
0.74
SPEARMAN RHO
Kelly Engine
BAYESIAN WR + REGIME-AWARE + CONSENSUS
Quarter-Kelly position sizing with Bayesian win-rate estimation. Source-type multipliers. E12 strategy: Oracle + NO + No Sports = 96.8% WR with 2x Kelly boost. 5-tier wallet classification.
96.8%
E12 WIN RATE
0.25x
KELLY FRACTION
RAG Advisor
QDRANT + 3-TIER SEMANTIC SEARCH
Retrieval-augmented generation over 20K+ intelligence vectors in Qdrant. Three collection tiers: current items, historical archive, and domain knowledge. Cosine similarity matching.
20K+
VECTORS
3
COLLECTIONS
LLM Scorer
CLAUDE HAIKU + GEMINI FLASH FALLBACK
AI-powered intelligence analysis producing executive briefings, narrative patterns, and Analysis of Competing Hypotheses. Dual-model redundancy for availability. Gated behind --llm flag.
2
LLM MODELS
ACH
METHOD
Geo Enrichment
131 REGEX PATTERNS + LAT/LON TAGGING
Content-based geographic identification. Extracts country, region, and coordinates from unstructured text. 85%+ coverage rate across all intelligence items.
131
GEO PATTERNS
85%+
COVERAGE
PM Edge Scorer
4-WAY DIVERGENCE + SOURCE CAPS
Cross-platform prediction market arbitrage. Polymarket vs Kalshi vs Manifold vs Metaculus divergence detection. Source-type edge caps: convergence 0.65, arbitrage 0.90, insider 0.95.
4
PLATFORMS
0.95
MAX EDGE CAP
Instability Scorer
6-TIER FRAMEWORK + ML BLEND
Political instability assessment across 16 watchlist countries. 70/30 heuristic/ML blend with structural floor. Global risk index computation. Six severity tiers from STABLE to CRITICAL.
16
COUNTRIES
6
SEVERITY TIERS
Entity Graph
CENTRALITY + COMMUNITIES + INSIGHTS
Pure Python entity relationship graph. Betweenness centrality, community detection, 5 insight types. Incremental merge for O(1) updates. Tracks entity co-occurrence across sources and time.
~3s
BUILD TIME
5
INSIGHT TYPES
SEC 04
Infrastructure
HETZNER CPX42 · 15 CONTAINERSpipeline
Data collection + processing
arena
Bot tournament engine
watcher
Oracle copy-trading daemon
dashboard
21-view SPA
data-api
REST API endpoints
postgres
Primary database
redis
Cache + message broker
qdrant
Vector database (20K+)
n8n
Workflow automation
nginx
Reverse proxy + SSL
grafana
Visualization + dashboards
prometheus
Metrics collection
cadvisor
Container metrics
node-exporter
Host metrics
pg-exporter
Postgres metrics
15/15
CONTAINERS UP
51
CRON JOBS
6
N8N WORKFLOWS
27
SHELL SCRIPTS
Self-Healing Loop
Automated container restart, pipeline trigger, stuck process recovery. 6 diagnostic check types with escalation. Zero manual intervention required for common failures.
EVERY 30 MIN
Health Diagnostics
7-point system health check: container status, pipeline freshness, arena cycle count, watcher heat level, disk usage, memory pressure, and cron health.
EVERY 2 HOURS
Alert System
Telegram push alerts for 9 event types: T1/T2 signals, bet recommendations, portfolio updates, instability alerts, weak signals, PM briefs, daily summaries, and market signals.
REAL-TIME
SEC 05
Data Flow & Storage
3-LAYER ARCHITECTURE~74
SECONDS / CYCLE
20-step pipeline, hourly
29
PARALLEL COLLECTORS
asyncio.gather, 6 modules
1,000-1,500
ITEMS / CYCLE
3,000 cap, 72h eviction
10-13
SIGNALS / CYCLE
T1/T2/T3 classified
20K+
QDRANT VECTORS
3 collections, cosine search
14K+
SQLITE RECORDS
7 tables, WAL mode, indexed
180
ARCHIVE RETENTION
JSONL daily archives
118
ORACLE WALLETS
5-tier classification
L1
Working Memory
JSON FILES
Hot storage for active intelligence. 3,000 item cap with 72-hour eviction. Immediate read/write access for pipeline processors and real-time queries.
3,000 items / 72h TTL
L2
Archive Layer
JSONL DAILY ARCHIVES
Compressed daily snapshots in append-only JSONL format. 180-day retention policy. Used for backtesting, trend analysis, and historical pattern matching.
180 days / daily rotation
L3
Analytics Store
SQLITE + WAL MODE
Structured analytics database with 7 indexed tables: items, signals, divergences, edges, trades, tipster snapshots, and metrics. Write-ahead logging for concurrent access.
14K+ records / 7 tables
SEC 06
Codebase Intelligence
159K LOC159,411
LINES OF CODE
Python, 300+ files
54+
PROCESSORS
ML, NLP, statistical, graph
172
API ENDPOINTS
REST + WebSocket
9
ALPHA MODELS
Statistical + ML hybrid
Technology Stack
RUNTIME
Python 3.13 + asyncio
Python 3.13 + asyncio
ML
scikit-learn + GradientBoosting
scikit-learn + GradientBoosting
VECTORS
Qdrant (3 collections)
Qdrant (3 collections)
DATABASE
PostgreSQL + SQLite + Redis
PostgreSQL + SQLite + Redis
BLOCKCHAIN
Polygon + py-clob-client
Polygon + py-clob-client
LLM
Claude Haiku + Gemini Flash
Claude Haiku + Gemini Flash
INFRA
Docker + Nginx + Let's Encrypt
Docker + Nginx + Let's Encrypt
MONITORING
Grafana + Prometheus + n8n
Grafana + Prometheus + n8n
ORACLE INTELLIGENCE NETWORK — CONFIDENTIAL
System Architecture Briefing — Generated
159,411 LOC · 29 Collectors · 51 Processors · 15 Containers · 20-Step Pipeline