charybdis/plugins/README.md

Charybdis Plugins

Plugins extend Charybdis with integrations to external tools (security scanners, issue trackers, identity providers). Plugins are compile-time integrated: they live as Rust crates under plugins/, declare protobuf extensions in plugins.toml, and are linked into the charybdis-server binary at build time.

This document covers:

• The plugin model (event-driven vs sync, traits, lifecycle)
• Configuring shipped plugins (DefectDojo, Keycloak)
• Writing a new plugin

Table of Contents

1. Plugin Types
2. Available Plugins
3. Plugin Configuration
4. Field Mapping System
5. Entity Resolution
6. DefectDojo Reference
7. Keycloak Reference
8. Writing a New Plugin
9. Plugin Lifecycle
10. Annotations
11. Troubleshooting

Plugin Types

Event-Driven Plugins

React to entity lifecycle events (Created, Updated, Deleted) — typically push data to an external system.

• Implement EventDrivenPlugin + one or more ResourceHandler per resource kind.
• Example: DefectDojo — Component creation → DefectDojo product + engagement.

Sync Plugins

Pull data from an external system on a schedule, creating or updating entities in Charybdis.

• Implement SyncPlugin (schedule, on_startup, manual_trigger).
• Example: Keycloak — periodic sync of users and groups.

Both types implement the base Plugin trait (name, validate_config, health_check).

Available Plugins

Plugin	Type	Status
DefectDojo	Event-driven	Shipped
Keycloak	Sync	Shipped

Planned integrations (Slack, Jira, GitHub, Dependency-Track) are tracked in VISION.md and TODO.md.

Plugin Configuration

Plugins are configured in config.toml under [plugins.<name>]. Use ${VAR} for secrets — never commit tokens.

[plugins.defectdojo]
enabled = true
base_url = "${DEFECTDOJO_URL}"
api_token = "${DEFECTDOJO_API_TOKEN}"

[plugins.keycloak]
enabled = true
base_url = "${KEYCLOAK_URL}"
realm = "charybdis"
client_id = "charybdis-sync"
client_secret = "${KEYCLOAK_CLIENT_SECRET}"

When enabled = false, the plugin is loaded but does not register handlers and does not react to events. See config.toml.example for the full reference.

Field Mapping System

Event-driven plugins (currently DefectDojo) use a configurable mapper to translate Charybdis entity fields into external-tool API payloads. Mappings are defined per resource type under [plugins.<name>.field_mappings.<resource>].

Three mapping types are supported:

1. Direct field mapping (string)

Dot-notation path into the entity.

name = "metadata.name"
description = "metadata.description"
email = "spec.profile.email"
tags = "metadata.tags"

2. Static value (object with value key)

business_criticality = { value = "high" }
is_active = { value = true }
priority = { value = 100 }

Supports strings, booleans, numbers, objects, and arrays.

3. Entity resolution (object)

Resolve a reference into a related entity and extract a field from it.

product_manager = {
  from = "spec.owner",
  resolve_entity = "User",
  extract = "annotations.defectdojo.com/user-id"
}

Parameter	Type	Required	Description
from	string	yes	Source field path
resolve_entity	string	yes	Entity kind to look up (User, Component, ...)
extract	string	yes	Field path to extract from the resolved entity
resolve_array	bool	no	Source field is an array of references
lookup_entity	string	no	Find an entity that references the source

Entity Resolution

Entity resolution lets a plugin navigate the entity graph at mapping time. The mapper performs a database lookup, extracts the target field, and substitutes it into the payload.

1. Source field            spec.owner = "user:john.doe"
2. Resolve entity          User entity for "john.doe"
3. Extract target field    annotations.defectdojo.com/user-id = "123"
4. Result                  product_manager = 123

Array resolution

user_ids = {
  from = "spec.members",
  resolve_entity = "User",
  extract = "annotations.defectdojo.com/user-id",
  resolve_array = true
}

Group.spec.members = ["user:john", "user:jane"] → user_ids = [123, 124].

Linked entity lookup

engagement_product = {
  from = "id",
  lookup_entity = "Component",
  extract = "annotations.defectdojo.com/product-id"
}

Finds a Component that references the source entity and extracts its annotation.

Error handling

Resolution failures (entity not found, missing annotation, invalid reference) log a warning and omit the field rather than failing the whole operation.

DefectDojo Reference

Maps Charybdis entities to DefectDojo resources via the REST API v2. Handlers shipped: ProductHandler, EngagementHandler, ProductTypeHandler, ProductMemberHandler.

Connection

[plugins.defectdojo]
enabled = true
base_url = "${DEFECTDOJO_URL}"
api_token = "${DEFECTDOJO_API_TOKEN}"
default_product_type_id = 1
auto_create_users = true
auto_create_product_types = false

To obtain the API token: in DefectDojo, User Profile → API Key.

Default engagement (auto-created with each product)

[plugins.defectdojo.default_engagement]
auto_create = true
name = "CI/CD Pipeline"
description = "Automated security scans from CI/CD pipeline"
engagement_type = "CI/CD"
status = "In Progress"
duration_days = 365
deduplication_on_engagement = true

Owner resolution

When a Component has an owner (typically a team name), Charybdis resolves it to DefectDojo users:

1. Find Group entity matching the owner.
2. Read Group.spec.members (usernames).
3. Look up each User entity.
4. Extract the user's email from the configured annotation.
5. Find the matching user in DefectDojo by email.

[plugins.defectdojo.owner_resolution]
user_email_annotation = "keycloak.com/email"   # depends on your IdP
defectdojo_lookup_field = "email"              # "email" or "username"
assign_all_members = true                       # all group members, or just first match

Product mapping (Component → DefectDojo Product)

[plugins.defectdojo.field_mappings.product]
name = "metadata.name"
description = "metadata.description"
product_type_id = { value = 1 }
tags = "metadata.tags"
business_criticality = { value = "high" }       # very high | high | medium | low | very low | none
platform = { value = "web" }                    # web | mobile | desktop | iot | ...
lifecycle = { value = "production" }
origin = { value = "internal" }
external_audience = { value = true }
internet_accessible = { value = true }

product_manager = {
  from = "spec.owner",
  resolve_entity = "User",
  extract = "annotations.defectdojo.com/user-id"
}

User mapping (User → DefectDojo User)

[plugins.defectdojo.field_mappings.user]
username = "metadata.name"
email = "spec.profile.email"
first_name = "spec.profile.displayName"
last_name = "spec.profile.displayName"
is_active = { value = true }

Product Type mapping (System → DefectDojo Product Type)

[plugins.defectdojo.field_mappings.product_type]
name = "metadata.name"
description = "metadata.description"
critical_product = { value = false }
key_product = { value = true }

Product Member mapping (Group → DefectDojo Product Member)

[plugins.defectdojo.field_mappings.product_member]
product_id = {
  from = "spec.parent",
  resolve_entity = "Component",
  extract = "annotations.defectdojo.com/product-id"
}

user_id = {
  from = "spec.members",
  resolve_entity = "User",
  extract = "annotations.defectdojo.com/user-id",
  resolve_array = true
}

role_name = { value = "Reader" }    # Owner | Maintainer | Writer | Reader | API_Importer

Engagement mapping (Resource → DefectDojo Engagement)

[plugins.defectdojo.field_mappings.engagement]
name = "metadata.name"
description = "metadata.description"

product_id = {
  from = "spec.owner",
  resolve_entity = "Component",
  extract = "annotations.defectdojo.com/product-id"
}

lead_id = {
  from = "spec.dependsOn",
  resolve_entity = "User",
  extract = "annotations.defectdojo.com/user-id"
}

target_start = "spec.target_start"
target_end = "spec.target_end"
status = { value = "In Progress" }
engagement_type = { value = "CI/CD" }
version = "metadata.annotations.version"
commit_hash = "metadata.annotations.commit_hash"
branch_tag = "metadata.annotations.branch"
build_id = "metadata.annotations.build_id"
source_code_management_uri = "metadata.annotations.repo_url"

deduplication_on_engagement = { value = true }
api_test = { value = true }
pen_test = { value = false }

Annotations written by the plugin

• defectdojo.com/product-id
• defectdojo.com/engagement-id
• defectdojo.com/owner-member-ids

Keycloak Reference

Sync plugin: pulls users and groups from a Keycloak realm via the Admin REST API on a schedule, on startup, or via manual trigger.

[plugins.keycloak]
enabled = true
base_url = "${KEYCLOAK_URL}"
realm = "charybdis"
client_id = "charybdis-sync"
client_secret = "${KEYCLOAK_CLIENT_SECRET}"

[plugins.keycloak.sync]
schedule = "0 */5 * * * *"   # optional cron; omit to sync only on startup/trigger
on_startup = true
manual_trigger = true
sync_users = true
sync_groups = true
namespace = "keycloak"
page_size = 100

The service account behind client_id/client_secret must have view-users and view-groups on the realm.

Synced entities carry annotations such as keycloak.com/email, keycloak.com/id, used by other plugins (e.g., the DefectDojo owner-resolution chain above).

Writing a New Plugin

1. Scaffold directory

mkdir -p plugins/my_plugin/{proto,src}

2. Register in plugins.toml

[plugins.my_plugin]
enabled = true
proto_path = "plugins/my_plugin/proto"
metadata_field_number = 103   # next available — see registry below
spec_field_number = 203
description = "My custom integration"

3. Define the protobuf schema

plugins/my_plugin/proto/my_plugin.proto:

syntax = "proto3";
package charybdis.plugins.my_plugin;

message MyPluginMetadata {
  string name = 1;
  string description = 2;
}

message MyPluginSpec {
  string integration_type = 1;
  bool enabled = 2;
}

build.rs regenerates proto/entities.proto automatically on next build.

4. Create the crate

plugins/my_plugin/Cargo.toml:

[package]
name = "charybdis-my-plugin-plugin"
version = "0.1.0"
edition = "2024"

[lib]
name = "charybdis_my_plugin"
path = "src/lib.rs"

[dependencies]
charybdis = { path = "../.." }
async-trait = "0.1"
anyhow = "1.0"
serde = { version = "1.0", features = ["derive"] }
tokio = { version = "1.48", features = ["full"] }
tracing = "0.1"

Add the crate to the workspace members in the top-level Cargo.toml.

5. Implement the plugin

Event-driven example:

use async_trait::async_trait;
use charybdis::plugins::{EventDrivenPlugin, Plugin, PluginConfig, PluginType, ResourceHandler};
use std::sync::Arc;

pub struct MyPlugin { /* config, client, handlers */ }

#[async_trait]
impl Plugin for MyPlugin {
    fn name(&self) -> &str { "my_plugin" }
    fn plugin_type(&self) -> PluginType { PluginType::EventDriven }
    fn load_config(&mut self, _: PluginConfig) -> anyhow::Result<()> { Ok(()) }
    fn validate_config(&self) -> anyhow::Result<()> { Ok(()) }
}

#[async_trait]
impl EventDrivenPlugin for MyPlugin {
    fn resource_handlers(&self) -> Vec<Arc<dyn ResourceHandler>> {
        vec![/* your handlers */]
    }
}

6. Register in charybdis-server

Add a loader block in charybdis-server/src/main.rs (follow the DefectDojo block as a template) and declare the dependency in charybdis-server/Cargo.toml.

Field number registry

Range	Allocation
1–99	Core entity types
100	DefectDojo
101	DependencyTrack
102	Keycloak
103+	Available

Field numbers MUST be unique across plugins — protobuf wire format depends on it.

Plugin Lifecycle

Build time

1. build.rs reads plugins.toml.
2. Generates proto/entities.proto with plugin-contributed oneof variants.
3. Compiles all protobuf files (tonic-prost-build).

Runtime

1. charybdis-server loads each enabled plugin from config.toml.
2. Event-driven plugins register their ResourceHandlers with the dispatcher.
3. Sync plugins register with the cron scheduler (and optionally run once at startup).
4. Entity CRUD operations publish EntityEvent to the in-memory event bus.
5. The dispatcher fans out events to matching handlers (filtered by trigger_kinds).
6. Handlers call external APIs and write annotations back to Charybdis.

Event flow

Client → gRPC CreateEntity → DB INSERT → EntityEvent::Created → Dispatcher
                                                                  ↓
                                          ResourceHandler.handle_create(entity)
                                                                  ↓
                                          POST /api/products → DefectDojo
                                                                  ↓
                                          update_annotations("defectdojo.com/product-id")

Annotations

Plugins store external IDs in entity annotations using reverse-DNS keys:

• defectdojo.com/product-id
• defectdojo.com/engagement-id
• keycloak.com/id
• keycloak.com/email

Query by annotation via JSONB in PostgreSQL:

SELECT * FROM entities
WHERE annotations->>'defectdojo.com/product-id' = '456';

Use EntityRepository::update_annotations() for atomic JSONB merge — it avoids the read-modify-write race when multiple handlers write to the same entity.

Troubleshooting

Plugin not reacting to events

• enabled = true in config.toml?
• Plugin registered in charybdis-server/src/main.rs?
• Entity kind matches the handler's trigger_kinds?
• Event bus running? (info!("Event bus started") in startup logs)

Field mapping not applied

• Path uses dot notation, e.g. metadata.name, not meta.name.
• Static value uses { value = "..." }, not a bare string when you want a literal.
• Field actually exists on the entity (inspect via the YAML adapter or GetEntity gRPC).

Entity resolution returns nothing

• Resolved entity exists in the DB.
• Resolved entity has the extract annotation set.
• resolve_entity uses the PascalCase kind ("User", not "user").
• For arrays, resolve_array = true is set.

DefectDojo API errors

• 400 Bad Request: Invalid product type ID → check default_product_type_id or your product_type_id mapping.
• 401 Unauthorized → check DEFECTDOJO_API_TOKEN.
• 409 Conflict: user already exists → expected, the plugin reuses existing users.

Performance

• Minimize entity-resolution chain depth.
• Prefer static values when the data isn't entity-bound.
• Check network latency to the external API — handlers run synchronously per event.

See Also

• Core Concepts — entity model, events, annotations.
• Architecture — protobuf schema, storage, scaling.
• Field Mapper Tests — reference behavior.