initial-commit

2026-05-12 17:06:43 +02:00
commit 051a080dfa
110 changed files with 26377 additions and 0 deletions
@@ -0,0 +1,340 @@
+# Charybdis Plugins
+
+This directory contains plugin implementations that extend Charybdis with integrations to external security and development tools.
+
+## What are Plugins?
+
+Charybdis supports **two types of plugins**, both compile-time integrated:
+
+### **1. Event-Driven Plugins**
+React to entity lifecycle events (create, update, delete):
+- **Example**: DefectDojo, DependencyTrack
+- **Implement**: `EventDrivenPlugin` trait + `ResourceHandler`
+- **Triggered by**: Entity CRUD operations
+- **Use case**: Auto-create resources in external tools when entities are created
+
+### **2. Sync Plugins**
+Pull data from external sources on a schedule:
+- **Example**: Okta, Keycloak, Active Directory
+- **Implement**: `SyncPlugin` trait
+- **Triggered by**: Cron schedule or manual API call
+- **Use case**: Sync users/groups from identity providers
+
+## Generic Utilities
+
+All plugins have access to reusable utilities in `src/plugins/`:
+
+- **`PluginHttpClient`** - Multi-auth HTTP client (Token, Bearer, API Key, Basic Auth)
+- **`AnnotationHelper`** - Consistent API for storing/retrieving plugin metadata
+- **`FieldMapper`** - Maps entity fields to external tool formats
+- **`DateUtils`** - Common date/time operations
+
+## Plugin Structure
+
+Each plugin follows this structure:
+
+```
+plugins/
+└── my_plugin/
+    ├── Cargo.toml              # Plugin crate definition
+    ├── README.md               # Plugin-specific documentation
+    ├── proto/
+    │   └── my_plugin.proto     # Protobuf entity definitions
+    └── src/
+        ├── lib.rs              # Plugin entry point
+        ├── handler.rs          # Event handler implementation
+        ├── client.rs           # External API client
+        └── config.rs           # Plugin configuration
+```
+
+## Available Plugins
+
+### DefectDojo
+- **Type**: Event-Driven
+- **Purpose**: Security vulnerability management integration
+- **External API**: DefectDojo REST API v2
+- **Handlers**: ProductHandler, EngagementHandler, ProductTypeHandler, ProductMemberHandler
+- **Features**:
+  - Automatic product creation on Component/Service creation
+  - Product updates on entity changes
+  - Auto-create CI/CD engagements
+  - Owner resolution (Group → members → email annotation → DefectDojo user lookup)
+  - Configurable OIDC provider mapping (Keycloak, Okta, Azure AD)
+  - Annotation storage (`defectdojo.com/product-id`, `defectdojo.com/engagement-id`, `defectdojo.com/owner-member-ids`)
+
+### Keycloak
+- **Type**: Sync
+- **Purpose**: Identity provider sync (users and groups)
+- **External API**: Keycloak Admin REST API
+- **Features**:
+  - User sync with profile and annotations (display_name, email, picture)
+  - Group sync with hierarchy (parent, children, members)
+  - Configurable annotations (e.g., `keycloak.com/email` for OIDC mapping)
+  - On-startup sync, manual trigger via gRPC
+
+### DependencyTrack (Scaffolded)
+- **Type**: Event-Driven
+- **Purpose**: Software supply chain security
+- **External API**: DependencyTrack REST API
+- **Status**: Scaffolded (proto + crate structure), handler logic not implemented
+- **Planned**:
+  - Auto-create projects for components
+  - SBOM ingestion
+
+## Creating a New Plugin
+
+### Quick Start
+
+1. **Create plugin directory structure:**
+   ```bash
+   mkdir -p plugins/my_plugin/{proto,src}
+   ```
+
+2. **Add to plugins.toml:**
+   ```toml
+   [plugins.my_plugin]
+   enabled = true
+   proto_path = "plugins/my_plugin/proto"
+   metadata_field_number = 102  # Use next available number
+   spec_field_number = 102
+   description = "My custom integration"
+   ```
+
+3. **Define protobuf schema:**
+   Create `plugins/my_plugin/proto/my_plugin.proto`:
+   ```protobuf
+   syntax = "proto3";
+   package charybdis.plugins.my_plugin;
+
+   message MyPluginMetadata {
+     string name = 1;
+     string description = 2;
+   }
+
+   message MyPluginSpec {
+     string integration_type = 1;
+     bool enabled = 2;
+   }
+   ```
+
+4. **Create plugin crate:**
+   Create `plugins/my_plugin/Cargo.toml`:
+   ```toml
+   [package]
+   name = "charybdis-plugin-my-plugin"
+   version = "0.1.0"
+   edition = "2021"
+
+   [dependencies]
+   charybdis = { path = "../.." }
+   async-trait = "0.1"
+   tokio = { version = "1.0", features = ["full"] }
+   tracing = "0.1"
+   ```
+
+5. **Implement event handler:**
+   Create `plugins/my_plugin/src/lib.rs`:
+   ```rust
+   use async_trait::async_trait;
+   use charybdis::events::{EventHandler, EntityEvent, EventResult};
+
+   pub struct MyPluginHandler;
+
+   #[async_trait]
+   impl EventHandler for MyPluginHandler {
+       async fn handle_event(&self, event: &EntityEvent) -> EventResult<()> {
+           // React to entity changes
+           Ok(())
+       }
+   }
+   ```
+
+6. **Build and test:**
+   ```bash
+   cargo build
+   cargo test
+   ```
+
+## Plugin Field Number Allocation
+
+Field numbers must be unique across all plugins to avoid protobuf conflicts:
+
+| Range    | Allocation            |
+|----------|-----------------------|
+| 1-99     | Core entity types     |
+| 100      | DefectDojo            |
+| 101      | DependencyTrack       |
+| 102-199  | Available for plugins |
+
+When creating a new plugin, use the next available number in the 102+ range.
+
+## Plugin Configuration
+
+Plugins are configured in `config.toml` with `${VAR}` env var substitution for secrets:
+
+```toml
+[plugins.defectdojo]
+enabled = true
+base_url = "${DEFECTDOJO_API_URL}"
+api_token = "${DEFECTDOJO_API_TOKEN}"
+
+[plugins.defectdojo.default_engagement]
+auto_create = true
+name = "CI/CD Pipeline"
+
+[plugins.keycloak]
+enabled = true
+base_url = "${KEYCLOAK_URL}"
+realm = "master"
+client_id = "charybdis-sync"
+client_secret = "${KEYCLOAK_CLIENT_SECRET}"
+```
+
+See `config.toml.example` for all options.
+
+## Plugin Lifecycle
+
+1. **Build Time:**
+   - `build.rs` reads `plugins.toml`
+   - Generates `proto/entities.proto` with plugin types
+   - Compiles all protobuf files
+   - Generates Rust code
+
+2. **Runtime:**
+   - Plugin handlers registered with event bus
+   - Events published on entity CRUD operations
+   - Handlers react asynchronously
+   - External APIs called as needed
+
+3. **Event Flow:**
+   ```
+   Client creates entity
+   → Entity stored in database
+   → Event published to event bus
+   → Plugin handler receives event
+   → Plugin calls external API
+   → Plugin stores external ID in annotations
+   ```
+
+## Using Annotations
+
+Plugins store external tool IDs in entity annotations:
+
+```rust
+// Store external ID
+entity.annotations.insert(
+    "my-plugin.com/resource-id".to_string(),
+    "ext-12345".to_string(),
+);
+
+// Query by external ID in SQL
+SELECT * FROM entities 
+WHERE annotations->>'my-plugin.com/resource-id' = 'ext-12345';
+```
+
+### Annotation Naming Convention
+
+Use reverse-DNS style:
+- `defectdojo.com/product-id`
+- `dependencytrack.com/project-uuid`
+- `github.com/repo-slug`
+- `{tool}.com/{resource}-{attribute}`
+
+## Best Practices
+
+### 1. Error Handling
+- Don't panic - return errors
+- Log but continue on non-critical failures
+- Implement retry logic for transient errors
+
+### 2. Idempotency
+- Check if resource exists before creating
+- Make operations safe to retry
+- Handle duplicate creation gracefully
+
+### 3. Performance
+- Don't block event handlers
+- Use `tokio::spawn` for long operations
+- Batch operations when possible
+
+### 4. Testing
+- Unit test event handlers
+- Mock external API clients
+- Integration tests with real APIs (optional)
+
+### 5. Documentation
+- Document required environment variables
+- Provide example configurations
+- Explain entity model and annotations
+
+## Contributing Plugins
+
+We welcome plugin contributions! To contribute:
+
+1. Fork the repository
+2. Create your plugin following the structure above
+3. Add comprehensive tests
+4. Document configuration and usage
+5. Submit a pull request
+
+### Plugin Requirements
+
+- [ ] Protobuf definitions with Metadata and Spec messages
+- [ ] Event handler implementation
+- [ ] External API client (if applicable)
+- [ ] Configuration via environment variables
+- [ ] README with setup instructions
+- [ ] Unit tests for event handler
+- [ ] Example usage in documentation
+
+## Plugin Distribution Models
+
+### In-Tree (Current)
+Plugins live in the `plugins/` directory and are enabled via `plugins.toml`.
+
+**Pros:**
+- Easy to discover
+- Consistent quality
+- Tested together
+
+**Cons:**
+- Requires core repo access
+- All plugins built together
+
+### External Crates (Future)
+Plugins distributed as separate Rust crates.
+
+**Example:**
+```toml
+[dependencies]
+charybdis-plugin-custom = "0.1"
+```
+
+**Pros:**
+- Independent versioning
+- Community contributions
+- Optional dependencies
+
+**Cons:**
+- Discovery harder
+- Compatibility challenges
+
+### Plugin Marketplace (Future)
+Central registry of available plugins (like Backstage).
+
+## Resources
+
+- [Plugin Configuration Guide](../docs/PLUGIN_CONFIGURATION_GUIDE.md)
+- [Core Concepts](../docs/core-concepts.md)
+- [Architecture](../docs/architecture.md)
+- [Protobuf Style Guide](https://protobuf.dev/programming-guides/style/)
+
+## Support
+
+- Open an issue for bug reports
+- Discussions for questions
+- PRs for contributions
+
+## License
+
+Same as Charybdis core (see LICENSE file in repository root)