Public Access
initial-commit
This commit is contained in:
Guillaume GRABÉ
@@ -0,0 +1,249 @@
|
||||
# Charybdis Vision
|
||||
|
||||
**The security-native platform engineering tool.**
|
||||
|
||||
## The Thesis
|
||||
|
||||
Backstage was built in 2020, at the dawn of microservices and Kubernetes adoption. It solved a real problem: "where do we catalog all these services?" But its approach — static YAML files, polling-based discovery, and a bolted-on plugin ecosystem — reflects the constraints of its era.
|
||||
|
||||
In 2026, the landscape has changed:
|
||||
|
||||
- **Infrastructure as Code is the norm.** Services are declared, not discovered. Events are emitted, not polled.
|
||||
- **Security and compliance are non-negotiable.** NIS2, DORA, SOC2, FedRAMP — every organization needs a clear picture of their security posture, not just a service catalog.
|
||||
- **Platform engineers want fewer tools, not more.** Running Backstage + DefectDojo + Dependency-Track + a license scanner + a compliance dashboard is unsustainable.
|
||||
- **The plugin ecosystem failed its promise.** Most Backstage plugins are thin wrappers, poorly maintained, and break between versions. The 1000+ plugin count is vanity — teams use 3 features.
|
||||
|
||||
Charybdis is what a platform engineering tool looks like when you start from these realities.
|
||||
|
||||
## What Charybdis Is
|
||||
|
||||
A **security-native platform engineering tool** that unifies software catalog, vulnerability management, and compliance posture in a single, event-driven platform.
|
||||
|
||||
Not "a catalog with security plugins." Not "a security dashboard with a catalog bolted on." A single tool where **every service in your catalog has its security posture, vulnerabilities, licenses, and compliance status as first-class data** — because they were never separate concerns to begin with.
|
||||
|
||||
### The Three Pillars
|
||||
|
||||
#### 1. Dynamic Software Catalog
|
||||
|
||||
Your software catalog should reflect reality, not YAML files that were accurate three months ago.
|
||||
|
||||
- **Event-driven** — Services register via gRPC from CI/CD pipelines, IaC tools, or Kubernetes controllers. No YAML files to maintain.
|
||||
- **Real-time** — Changes propagate instantly to all consumers. No polling, no stale data.
|
||||
- **Rich entity model** — Components, Systems, APIs, Users, Groups, Domains, Resources. Compatible with Backstage's descriptor format for migration.
|
||||
|
||||
#### 2. Native Security Posture
|
||||
|
||||
Every entity in the catalog carries its security context natively.
|
||||
|
||||
- **Vulnerability management** — Ingest scan results (SARIF, CycloneDX, SPDX) directly. No external vuln management tool needed.
|
||||
- **Assessment workflow** — Triage, accept risk, or remediate. Rules engine for auto-assessment based on severity, component, scanner.
|
||||
- **Security gates** — Define thresholds per product. Block deployments when critical vulnerabilities exceed limits.
|
||||
- **License compliance** — Track licenses across your dependency tree. Enforce policies. Flag violations.
|
||||
|
||||
### Future: Scaffolder
|
||||
|
||||
A modern scaffolding system that goes beyond template rendering:
|
||||
|
||||
- **Event-driven provisioning** — Create a service from a template and the entire toolchain is provisioned automatically: repo, CI/CD, security scanning, monitoring, catalog entry.
|
||||
- **Git-native templates** — No Nunjucks. Templates are real repositories with real code.
|
||||
- **Policy-driven** — Templates enforce organizational standards by default.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
### For Platform Engineers
|
||||
|
||||
**Before Charybdis:**
|
||||
```
|
||||
New service → Create repo → Add catalog-info.yaml → PR to catalog →
|
||||
Wait for merge → Wait for Backstage poll → Manually create DefectDojo product →
|
||||
Manually create DT project → Manually configure scanner → Hope someone updates
|
||||
the YAML when things change
|
||||
```
|
||||
|
||||
**With Charybdis:**
|
||||
```
|
||||
New service → One gRPC call → Cataloged, security scanning configured,
|
||||
vulnerability tracking active, compliance monitored. Real-time. Always accurate.
|
||||
```
|
||||
|
||||
### For Security Engineers
|
||||
|
||||
**Before Charybdis:**
|
||||
- Vulnerability data scattered across DefectDojo, Dependency-Track, Snyk, SonarQube
|
||||
- No link between "this service" and "its vulnerabilities"
|
||||
- Compliance evidence assembled manually from 5 different tools
|
||||
- Security posture visibility requires stitching together multiple dashboards
|
||||
|
||||
**With Charybdis:**
|
||||
- One dashboard: every service, its vulnerabilities, its licenses, its compliance status
|
||||
- Native scan ingestion — SARIF covers 60%+ of modern scanners
|
||||
- Assessment workflows built-in, not bolted on
|
||||
- Compliance frameworks (NIS2, SOC2) mapped to actual vulnerability data
|
||||
|
||||
### For Engineering Leadership
|
||||
|
||||
- **Single pane of glass** for software inventory AND security posture
|
||||
- **Compliance reporting** that pulls from real data, not spreadsheets
|
||||
- **Risk visibility** per service, per team, per domain
|
||||
- **One tool to deploy and maintain** instead of a fragmented toolchain
|
||||
|
||||
## Technical Differentiators
|
||||
|
||||
| | Backstage | Charybdis |
|
||||
|---|---|---|
|
||||
| **Architecture** | Static YAML + polling | Event-driven + gRPC |
|
||||
| **Catalog updates** | PR → merge → poll (minutes to hours) | API call → instant |
|
||||
| **Security** | Plugin ecosystem (fragmented) | Native (first-class) |
|
||||
| **Deployment** | Node.js cluster + PostgreSQL + plugins | Single Rust binary + PostgreSQL |
|
||||
| **Performance** | Degrades at scale (>5k entities) | Sub-millisecond p99, tested at 170k entities |
|
||||
| **Plugin quality** | Variable (many abandoned) | Core features native, integrations as plugins |
|
||||
| **Language** | TypeScript | Rust (memory-safe, high-performance) |
|
||||
| **Resource usage** | ~1GB+ RAM | ~50MB RAM |
|
||||
| **Vulnerability management** | Requires external tools | Built-in |
|
||||
| **Compliance** | Manual / external | Native frameworks |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Charybdis
|
||||
┌─────────────────────────────────────────┐
|
||||
│ │
|
||||
CI/CD ──gRPC──────▶│ Software Catalog (event-driven) │
|
||||
IaC tools ────────▶│ Vulnerability Management (native) │
|
||||
K8s controllers ──▶│ License Compliance (native) │
|
||||
Scanners ─────────▶│ Assessment & Rules Engine │
|
||||
│ Security Gates │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────┐ │
|
||||
│ │ Event Bus │ │
|
||||
│ │ EntityCreated → Plugins react │ │
|
||||
│ │ VulnIngested → Rules evaluate │ │
|
||||
│ │ GateFailed → Notifications fire │ │
|
||||
│ └─────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────┐ │
|
||||
│ │ Plugins (integrations only) │ │
|
||||
│ │ Jira · Slack · Teams · GitHub │ │
|
||||
│ │ GitLab · PagerDuty · Custom │ │
|
||||
│ └─────────────────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key insight:** Security features are in the core, not in plugins. Plugins handle integrations with external systems (notifications, issue trackers). This is the opposite of Backstage's model.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 0: Foundation (Done)
|
||||
- [x] gRPC API with full entity CRUD
|
||||
- [x] PostgreSQL storage (zero-migration, protobuf + JSONB)
|
||||
- [x] Event bus system
|
||||
- [x] Plugin framework (EventDriven + Sync)
|
||||
- [x] mTLS + RBAC security
|
||||
- [x] OpenTelemetry observability
|
||||
- [x] Backstage YAML adapter
|
||||
- [x] DefectDojo plugin (products, engagements, owner resolution)
|
||||
- [x] Keycloak plugin (user/group sync with annotations)
|
||||
|
||||
### Phase 1: Security Core
|
||||
|
||||
Native vulnerability management — the feature that makes Charybdis a DefectDojo/Dependency-Track replacement, not just a catalog.
|
||||
|
||||
#### Architecture: Core + Extensible Parsers
|
||||
|
||||
Security features live in the **core**, not in plugins. Plugins remain for external integrations (Slack, Jira, DefectDojo sync). The parser system is extensible without touching reconciliation logic.
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────────┐
|
||||
│ gRPC IngestionService │
|
||||
│ - ImportScan(component, lifecycle, format, data, options) │
|
||||
│ - DryRunScan(component, lifecycle, format, data) │
|
||||
└───────────────────────────┬────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────────────────▼────────────────────────────────────┐
|
||||
│ Parser Registry (extensible) │
|
||||
│ - ScannerParser trait │
|
||||
│ - Built-in: SARIF, CycloneDX, ... │
|
||||
│ - Plugin-contributed: exotic formats via contributed_parsers()│
|
||||
└───────────────────────────┬────────────────────────────────────┘
|
||||
│ Vec<NormalizedFinding>
|
||||
┌───────────────────────────▼────────────────────────────────────┐
|
||||
│ Reconciliation Engine (core) │
|
||||
│ - Fingerprint calculation (scanner + rule_id + file + line) │
|
||||
│ - Match existing findings by (component_id, lifecycle) │
|
||||
│ - Produce diff: New / Unchanged / Resolved / Reopened │
|
||||
└───────────────────────────┬────────────────────────────────────┘
|
||||
│ ReconciliationResult
|
||||
┌───────────────────────────▼────────────────────────────────────┐
|
||||
│ Lifecycle Manager (core) │
|
||||
│ - Apply: create new, update last_seen, close resolved │
|
||||
│ - DryRun: return diff without persisting │
|
||||
│ - Publish events (FindingCreated, FindingResolved, etc.) │
|
||||
└────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Key design decisions:**
|
||||
- **Parsers are extensible, reconciliation is core.** Adding a scanner = implement one trait (~50-100 lines). Dedup/close logic is written and tested once.
|
||||
- **SARIF as primary format.** Covers 70%+ of modern scanners (Trivy, Semgrep, CodeQL, Checkov, etc.) with zero per-scanner code.
|
||||
- **Findings scoped to (component, lifecycle).** Allows per-environment tracking (prod vs integration) without duplicating catalog entities.
|
||||
- **Dry-run is native.** Enables MR-level diff: "this MR introduces X new vulnerabilities" — a feature DefectDojo still doesn't have.
|
||||
|
||||
#### Milestones
|
||||
|
||||
- [ ] Finding entity kind (proto + storage)
|
||||
- [ ] IngestionService gRPC endpoint (ImportScan + DryRun)
|
||||
- [ ] ScannerParser trait + ParserRegistry
|
||||
- [ ] SARIF parser (built-in, covers majority of modern scanners)
|
||||
- [ ] Fingerprint-based deduplication
|
||||
- [ ] Reconciliation engine (new/unchanged/resolved/reopened)
|
||||
- [ ] CycloneDX VEX parser
|
||||
- [ ] Assessment workflow (triage, accept, remediate)
|
||||
- [ ] Rules engine for auto-assessment
|
||||
- [ ] Security gates (severity thresholds per product)
|
||||
|
||||
### Phase 2: Compliance & Integrations
|
||||
- [ ] Compliance framework mappings (NIS2, SOC2, DORA)
|
||||
- [ ] License policy engine
|
||||
- [ ] VEX document support (CSAF, OpenVEX)
|
||||
- [ ] Export/reporting (PDF, Excel)
|
||||
- [ ] Notification plugins (Slack, Teams, email)
|
||||
- [ ] Issue tracker plugins (Jira, GitHub, GitLab)
|
||||
|
||||
### Phase 3: Scaffolder & Ecosystem
|
||||
- [ ] Service scaffolder (Git-native templates)
|
||||
- [ ] Event-driven provisioning on scaffold
|
||||
- [ ] Plugin SDK documentation
|
||||
- [ ] Community plugin registry
|
||||
- [ ] Helm chart & 1-click deploy
|
||||
|
||||
## Target Audience
|
||||
|
||||
**Primary:** Platform engineering teams and security engineers at companies with 50-5000 engineers who need both a software catalog and security posture visibility, and are tired of stitching together 5+ tools.
|
||||
|
||||
**Secondary:** Organizations evaluating Backstage but hesitant about the operational complexity, or currently running Backstage and frustrated with stale data and plugin maintenance.
|
||||
|
||||
## Open Source Strategy
|
||||
|
||||
Charybdis is and will remain **fully open source** (Apache-2.0).
|
||||
|
||||
The value proposition is clear enough that adoption will be driven by the product itself:
|
||||
- Zero-cost alternative to commercial platforms (Port, Cortex, OpsLevel)
|
||||
- Dramatically simpler than self-hosting Backstage + security tools
|
||||
- Security-native approach that no other open-source tool offers
|
||||
|
||||
Community growth will come from:
|
||||
1. **Security engineers** frustrated with fragmented tooling
|
||||
2. **Platform engineers** looking for a lighter alternative to Backstage
|
||||
3. **Small-to-mid companies** that can't justify 5 separate tools
|
||||
4. **Compliance-driven organizations** that need integrated security posture
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- **Not a SIEM.** Charybdis manages software catalog and vulnerability posture, not security events or incident response.
|
||||
- **Not a scanner.** Charybdis ingests scan results. It doesn't run scanners itself. Use your existing scanners (Trivy, Semgrep, ZAP, etc.) and push results to Charybdis.
|
||||
- **Not a CI/CD platform.** Charybdis integrates with your CI/CD. It doesn't replace it.
|
||||
- **Not trying to have 1000 plugins.** Core features are native. Plugins are for integrations with external systems. Quality over quantity.
|
||||
|
||||
---
|
||||
|
||||
*Built with Rust. Secured by design. One binary to rule them all.*
|
||||
Reference in New Issue
Block a user