Public Access
340 lines
8.9 KiB
Markdown
340 lines
8.9 KiB
Markdown
# Charybdis Plugins
|
|
|
|
This directory contains plugin implementations that extend Charybdis with integrations to external security and development tools.
|
|
|
|
## What are Plugins?
|
|
|
|
Charybdis supports **two types of plugins**, both compile-time integrated:
|
|
|
|
### **1. Event-Driven Plugins**
|
|
React to entity lifecycle events (create, update, delete):
|
|
- **Example**: DefectDojo, DependencyTrack
|
|
- **Implement**: `EventDrivenPlugin` trait + `ResourceHandler`
|
|
- **Triggered by**: Entity CRUD operations
|
|
- **Use case**: Auto-create resources in external tools when entities are created
|
|
|
|
### **2. Sync Plugins**
|
|
Pull data from external sources on a schedule:
|
|
- **Example**: Okta, Keycloak, Active Directory
|
|
- **Implement**: `SyncPlugin` trait
|
|
- **Triggered by**: Cron schedule or manual API call
|
|
- **Use case**: Sync users/groups from identity providers
|
|
|
|
## Generic Utilities
|
|
|
|
All plugins have access to reusable utilities in `src/plugins/`:
|
|
|
|
- **`PluginHttpClient`** - Multi-auth HTTP client (Token, Bearer, API Key, Basic Auth)
|
|
- **`AnnotationHelper`** - Consistent API for storing/retrieving plugin metadata
|
|
- **`FieldMapper`** - Maps entity fields to external tool formats
|
|
- **`DateUtils`** - Common date/time operations
|
|
|
|
## Plugin Structure
|
|
|
|
Each plugin follows this structure:
|
|
|
|
```
|
|
plugins/
|
|
└── my_plugin/
|
|
├── Cargo.toml # Plugin crate definition
|
|
├── README.md # Plugin-specific documentation
|
|
├── proto/
|
|
│ └── my_plugin.proto # Protobuf entity definitions
|
|
└── src/
|
|
├── lib.rs # Plugin entry point
|
|
├── handler.rs # Event handler implementation
|
|
├── client.rs # External API client
|
|
└── config.rs # Plugin configuration
|
|
```
|
|
|
|
## Available Plugins
|
|
|
|
### DefectDojo
|
|
- **Type**: Event-Driven
|
|
- **Purpose**: Security vulnerability management integration
|
|
- **External API**: DefectDojo REST API v2
|
|
- **Handlers**: ProductHandler, EngagementHandler, ProductTypeHandler, ProductMemberHandler
|
|
- **Features**:
|
|
- Automatic product creation on Component/Service creation
|
|
- Product updates on entity changes
|
|
- Auto-create CI/CD engagements
|
|
- Owner resolution (Group → members → email annotation → DefectDojo user lookup)
|
|
- Configurable OIDC provider mapping (Keycloak, Okta, Azure AD)
|
|
- Annotation storage (`defectdojo.com/product-id`, `defectdojo.com/engagement-id`, `defectdojo.com/owner-member-ids`)
|
|
|
|
### Keycloak
|
|
- **Type**: Sync
|
|
- **Purpose**: Identity provider sync (users and groups)
|
|
- **External API**: Keycloak Admin REST API
|
|
- **Features**:
|
|
- User sync with profile and annotations (display_name, email, picture)
|
|
- Group sync with hierarchy (parent, children, members)
|
|
- Configurable annotations (e.g., `keycloak.com/email` for OIDC mapping)
|
|
- On-startup sync, manual trigger via gRPC
|
|
|
|
### DependencyTrack (Scaffolded)
|
|
- **Type**: Event-Driven
|
|
- **Purpose**: Software supply chain security
|
|
- **External API**: DependencyTrack REST API
|
|
- **Status**: Scaffolded (proto + crate structure), handler logic not implemented
|
|
- **Planned**:
|
|
- Auto-create projects for components
|
|
- SBOM ingestion
|
|
|
|
## Creating a New Plugin
|
|
|
|
### Quick Start
|
|
|
|
1. **Create plugin directory structure:**
|
|
```bash
|
|
mkdir -p plugins/my_plugin/{proto,src}
|
|
```
|
|
|
|
2. **Add to plugins.toml:**
|
|
```toml
|
|
[plugins.my_plugin]
|
|
enabled = true
|
|
proto_path = "plugins/my_plugin/proto"
|
|
metadata_field_number = 102 # Use next available number
|
|
spec_field_number = 102
|
|
description = "My custom integration"
|
|
```
|
|
|
|
3. **Define protobuf schema:**
|
|
Create `plugins/my_plugin/proto/my_plugin.proto`:
|
|
```protobuf
|
|
syntax = "proto3";
|
|
package charybdis.plugins.my_plugin;
|
|
|
|
message MyPluginMetadata {
|
|
string name = 1;
|
|
string description = 2;
|
|
}
|
|
|
|
message MyPluginSpec {
|
|
string integration_type = 1;
|
|
bool enabled = 2;
|
|
}
|
|
```
|
|
|
|
4. **Create plugin crate:**
|
|
Create `plugins/my_plugin/Cargo.toml`:
|
|
```toml
|
|
[package]
|
|
name = "charybdis-plugin-my-plugin"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
|
|
[dependencies]
|
|
charybdis = { path = "../.." }
|
|
async-trait = "0.1"
|
|
tokio = { version = "1.0", features = ["full"] }
|
|
tracing = "0.1"
|
|
```
|
|
|
|
5. **Implement event handler:**
|
|
Create `plugins/my_plugin/src/lib.rs`:
|
|
```rust
|
|
use async_trait::async_trait;
|
|
use charybdis::events::{EventHandler, EntityEvent, EventResult};
|
|
|
|
pub struct MyPluginHandler;
|
|
|
|
#[async_trait]
|
|
impl EventHandler for MyPluginHandler {
|
|
async fn handle_event(&self, event: &EntityEvent) -> EventResult<()> {
|
|
// React to entity changes
|
|
Ok(())
|
|
}
|
|
}
|
|
```
|
|
|
|
6. **Build and test:**
|
|
```bash
|
|
cargo build
|
|
cargo test
|
|
```
|
|
|
|
## Plugin Field Number Allocation
|
|
|
|
Field numbers must be unique across all plugins to avoid protobuf conflicts:
|
|
|
|
| Range | Allocation |
|
|
|----------|-----------------------|
|
|
| 1-99 | Core entity types |
|
|
| 100 | DefectDojo |
|
|
| 101 | DependencyTrack |
|
|
| 102-199 | Available for plugins |
|
|
|
|
When creating a new plugin, use the next available number in the 102+ range.
|
|
|
|
## Plugin Configuration
|
|
|
|
Plugins are configured in `config.toml` with `${VAR}` env var substitution for secrets:
|
|
|
|
```toml
|
|
[plugins.defectdojo]
|
|
enabled = true
|
|
base_url = "${DEFECTDOJO_API_URL}"
|
|
api_token = "${DEFECTDOJO_API_TOKEN}"
|
|
|
|
[plugins.defectdojo.default_engagement]
|
|
auto_create = true
|
|
name = "CI/CD Pipeline"
|
|
|
|
[plugins.keycloak]
|
|
enabled = true
|
|
base_url = "${KEYCLOAK_URL}"
|
|
realm = "master"
|
|
client_id = "charybdis-sync"
|
|
client_secret = "${KEYCLOAK_CLIENT_SECRET}"
|
|
```
|
|
|
|
See `config.toml.example` for all options.
|
|
|
|
## Plugin Lifecycle
|
|
|
|
1. **Build Time:**
|
|
- `build.rs` reads `plugins.toml`
|
|
- Generates `proto/entities.proto` with plugin types
|
|
- Compiles all protobuf files
|
|
- Generates Rust code
|
|
|
|
2. **Runtime:**
|
|
- Plugin handlers registered with event bus
|
|
- Events published on entity CRUD operations
|
|
- Handlers react asynchronously
|
|
- External APIs called as needed
|
|
|
|
3. **Event Flow:**
|
|
```
|
|
Client creates entity
|
|
→ Entity stored in database
|
|
→ Event published to event bus
|
|
→ Plugin handler receives event
|
|
→ Plugin calls external API
|
|
→ Plugin stores external ID in annotations
|
|
```
|
|
|
|
## Using Annotations
|
|
|
|
Plugins store external tool IDs in entity annotations:
|
|
|
|
```rust
|
|
// Store external ID
|
|
entity.annotations.insert(
|
|
"my-plugin.com/resource-id".to_string(),
|
|
"ext-12345".to_string(),
|
|
);
|
|
|
|
// Query by external ID in SQL
|
|
SELECT * FROM entities
|
|
WHERE annotations->>'my-plugin.com/resource-id' = 'ext-12345';
|
|
```
|
|
|
|
### Annotation Naming Convention
|
|
|
|
Use reverse-DNS style:
|
|
- `defectdojo.com/product-id`
|
|
- `dependencytrack.com/project-uuid`
|
|
- `github.com/repo-slug`
|
|
- `{tool}.com/{resource}-{attribute}`
|
|
|
|
## Best Practices
|
|
|
|
### 1. Error Handling
|
|
- Don't panic - return errors
|
|
- Log but continue on non-critical failures
|
|
- Implement retry logic for transient errors
|
|
|
|
### 2. Idempotency
|
|
- Check if resource exists before creating
|
|
- Make operations safe to retry
|
|
- Handle duplicate creation gracefully
|
|
|
|
### 3. Performance
|
|
- Don't block event handlers
|
|
- Use `tokio::spawn` for long operations
|
|
- Batch operations when possible
|
|
|
|
### 4. Testing
|
|
- Unit test event handlers
|
|
- Mock external API clients
|
|
- Integration tests with real APIs (optional)
|
|
|
|
### 5. Documentation
|
|
- Document required environment variables
|
|
- Provide example configurations
|
|
- Explain entity model and annotations
|
|
|
|
## Contributing Plugins
|
|
|
|
We welcome plugin contributions! To contribute:
|
|
|
|
1. Fork the repository
|
|
2. Create your plugin following the structure above
|
|
3. Add comprehensive tests
|
|
4. Document configuration and usage
|
|
5. Submit a pull request
|
|
|
|
### Plugin Requirements
|
|
|
|
- [ ] Protobuf definitions with Metadata and Spec messages
|
|
- [ ] Event handler implementation
|
|
- [ ] External API client (if applicable)
|
|
- [ ] Configuration via environment variables
|
|
- [ ] README with setup instructions
|
|
- [ ] Unit tests for event handler
|
|
- [ ] Example usage in documentation
|
|
|
|
## Plugin Distribution Models
|
|
|
|
### In-Tree (Current)
|
|
Plugins live in the `plugins/` directory and are enabled via `plugins.toml`.
|
|
|
|
**Pros:**
|
|
- Easy to discover
|
|
- Consistent quality
|
|
- Tested together
|
|
|
|
**Cons:**
|
|
- Requires core repo access
|
|
- All plugins built together
|
|
|
|
### External Crates (Future)
|
|
Plugins distributed as separate Rust crates.
|
|
|
|
**Example:**
|
|
```toml
|
|
[dependencies]
|
|
charybdis-plugin-custom = "0.1"
|
|
```
|
|
|
|
**Pros:**
|
|
- Independent versioning
|
|
- Community contributions
|
|
- Optional dependencies
|
|
|
|
**Cons:**
|
|
- Discovery harder
|
|
- Compatibility challenges
|
|
|
|
### Plugin Marketplace (Future)
|
|
Central registry of available plugins (like Backstage).
|
|
|
|
## Resources
|
|
|
|
- [Plugin Configuration Guide](../docs/PLUGIN_CONFIGURATION_GUIDE.md)
|
|
- [Core Concepts](../docs/core-concepts.md)
|
|
- [Architecture](../docs/architecture.md)
|
|
- [Protobuf Style Guide](https://protobuf.dev/programming-guides/style/)
|
|
|
|
## Support
|
|
|
|
- Open an issue for bug reports
|
|
- Discussions for questions
|
|
- PRs for contributions
|
|
|
|
## License
|
|
|
|
Same as Charybdis core (see LICENSE file in repository root) |