Public Access
initial-commit
This commit is contained in:
Guillaume GRABÉ
@@ -0,0 +1,508 @@
|
||||
# Getting Started with Charybdis
|
||||
|
||||
This guide will get you from zero to a running Charybdis instance in minutes. By the end, you'll have a working software catalog that can register services via gRPC, ingest scan results, and auto-trigger integrations via event-driven plugins.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Rust** - 1.70 or later ([install](https://rustup.rs/))
|
||||
- **PostgreSQL** - 14 or later
|
||||
- **grpcurl** - For testing (optional, [install](https://github.com/fullstorydev/grpcurl))
|
||||
|
||||
## Installation
|
||||
|
||||
### Option 1: From Source
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/charybdis-catalog/charybdis.git
|
||||
cd charybdis
|
||||
|
||||
# Build
|
||||
cargo build --release
|
||||
|
||||
# The binary will be at target/release/charybdis
|
||||
```
|
||||
|
||||
### Option 2: Docker (Coming Soon)
|
||||
|
||||
```bash
|
||||
docker pull charybdis/charybdis:latest
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Start PostgreSQL
|
||||
|
||||
Using Docker:
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--name charybdis-postgres \
|
||||
-e POSTGRES_PASSWORD=mysecretpassword \
|
||||
-p 5432:5432 \
|
||||
postgres:15
|
||||
```
|
||||
|
||||
Or use an existing PostgreSQL instance.
|
||||
|
||||
### Step 2: Configure Charybdis
|
||||
|
||||
**Recommended: Use config.toml**
|
||||
|
||||
Copy the example configuration:
|
||||
|
||||
```bash
|
||||
cp config.toml.example config.toml
|
||||
```
|
||||
|
||||
Edit `config.toml` and set your database URL:
|
||||
|
||||
```toml
|
||||
[database]
|
||||
url = "${DATABASE_URL}"
|
||||
```
|
||||
|
||||
Set the environment variable:
|
||||
|
||||
```bash
|
||||
export DATABASE_URL="postgresql://postgres:mysecretpassword@localhost:5432/postgres"
|
||||
```
|
||||
|
||||
**Alternative: Environment Variables Only (Legacy)**
|
||||
|
||||
If you prefer environment variables:
|
||||
|
||||
```bash
|
||||
# Database
|
||||
export DATABASE_URL="postgresql://postgres:mysecretpassword@localhost:5432/postgres"
|
||||
|
||||
# Disable security for quick start
|
||||
export SECURITY_MTLS_ENABLED=false
|
||||
export SECURITY_RBAC_ENABLED=false
|
||||
|
||||
# Logging
|
||||
export RUST_LOG=info,charybdis=debug
|
||||
```
|
||||
|
||||
### Step 3: Run Charybdis
|
||||
|
||||
```bash
|
||||
cargo run
|
||||
```
|
||||
|
||||
You should see:
|
||||
|
||||
```
|
||||
INFO charybdis: Database ready
|
||||
INFO charybdis: Event bus started successfully
|
||||
INFO charybdis: EntityService server listening on [::1]:50051
|
||||
```
|
||||
|
||||
### Step 4: Verify It's Working
|
||||
|
||||
Test with grpcurl:
|
||||
|
||||
```bash
|
||||
# List available services
|
||||
grpcurl -plaintext localhost:50051 list
|
||||
|
||||
# Output:
|
||||
# charybdis.entities.EntityService
|
||||
# grpc.reflection.v1.ServerReflection
|
||||
```
|
||||
|
||||
Congratulations! Charybdis is running! 🎉
|
||||
|
||||
## Creating Your First Entity
|
||||
|
||||
### Using grpcurl
|
||||
|
||||
Create a service entity:
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext \
|
||||
-d '{
|
||||
"entity": {
|
||||
"kind": "Service",
|
||||
"service_metadata": {
|
||||
"name": "payment-service",
|
||||
"namespace": "production",
|
||||
"description": "Core payment processing service"
|
||||
}
|
||||
}
|
||||
}' \
|
||||
localhost:50051 charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"entity": {
|
||||
"id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"kind": "Service",
|
||||
"serviceMetadata": {
|
||||
"name": "payment-service",
|
||||
"namespace": "production",
|
||||
"description": "Core payment processing service"
|
||||
},
|
||||
"createdAt": "2025-11-04T10:00:00Z",
|
||||
"updatedAt": "2025-11-04T10:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### List All Entities
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext -d '{}' \
|
||||
localhost:50051 charybdis.entities.EntityService/ListEntities
|
||||
```
|
||||
|
||||
### Get Entity by ID
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext \
|
||||
-d '{"id": "550e8400-e29b-41d4-a716-446655440000"}' \
|
||||
localhost:50051 charybdis.entities.EntityService/GetEntity
|
||||
```
|
||||
|
||||
## Entity Types
|
||||
|
||||
Charybdis supports three main entity types:
|
||||
|
||||
### Service
|
||||
|
||||
Individual microservices or applications:
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext -d '{
|
||||
"entity": {
|
||||
"kind": "Service",
|
||||
"service_metadata": {
|
||||
"name": "user-api",
|
||||
"namespace": "production",
|
||||
"description": "User management API"
|
||||
}
|
||||
}
|
||||
}' localhost:50051 charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
### System
|
||||
|
||||
Collections of related services:
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext -d '{
|
||||
"entity": {
|
||||
"kind": "System",
|
||||
"system_metadata": {
|
||||
"name": "payment-system",
|
||||
"namespace": "production",
|
||||
"description": "Complete payment processing system"
|
||||
}
|
||||
}
|
||||
}' localhost:50051 charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
### Component
|
||||
|
||||
Reusable components or libraries:
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext -d '{
|
||||
"entity": {
|
||||
"kind": "Component",
|
||||
"component_spec": {
|
||||
"type": "library",
|
||||
"lifecycle": "production",
|
||||
"owner": "platform-team"
|
||||
},
|
||||
"component_metadata": {
|
||||
"name": "auth-library",
|
||||
"namespace": "shared",
|
||||
"description": "Shared authentication library"
|
||||
}
|
||||
}
|
||||
}' localhost:50051 charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
## Adding Metadata
|
||||
|
||||
Entities support rich metadata:
|
||||
|
||||
```bash
|
||||
grpcurl -plaintext -d '{
|
||||
"entity": {
|
||||
"kind": "Service",
|
||||
"service_metadata": {
|
||||
"name": "payment-service",
|
||||
"namespace": "production",
|
||||
"description": "Payment processing",
|
||||
"labels": {
|
||||
"team": "payments",
|
||||
"tier": "critical"
|
||||
},
|
||||
"links": [
|
||||
{
|
||||
"url": "https://dashboard.company.com/payments",
|
||||
"title": "Dashboard",
|
||||
"icon": "dashboard"
|
||||
}
|
||||
],
|
||||
"tags": ["payments", "pci-compliant", "critical"]
|
||||
},
|
||||
"annotations": {
|
||||
"github.com/repo-slug": "myorg/payment-service",
|
||||
"pagerduty.com/service-id": "PXYZ123"
|
||||
}
|
||||
}
|
||||
}' localhost:50051 charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
## Integrating with CI/CD
|
||||
|
||||
### Example: GitHub Actions
|
||||
|
||||
Create a workflow to register services automatically:
|
||||
|
||||
```yaml
|
||||
name: Register Service
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
register:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Register in Charybdis
|
||||
run: |
|
||||
grpcurl -plaintext \
|
||||
-d '{
|
||||
"entity": {
|
||||
"kind": "Service",
|
||||
"service_metadata": {
|
||||
"name": "${{ github.event.repository.name }}",
|
||||
"namespace": "production",
|
||||
"description": "${{ github.event.repository.description }}"
|
||||
},
|
||||
"annotations": {
|
||||
"github.com/repo-slug": "${{ github.repository }}"
|
||||
}
|
||||
}
|
||||
}' \
|
||||
your-charybdis-host:50051 \
|
||||
charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
### Example: GitLab CI
|
||||
|
||||
```yaml
|
||||
register_service:
|
||||
stage: deploy
|
||||
script:
|
||||
- |
|
||||
grpcurl -plaintext \
|
||||
-d "{
|
||||
\"entity\": {
|
||||
\"kind\": \"Service\",
|
||||
\"service_metadata\": {
|
||||
\"name\": \"${CI_PROJECT_NAME}\",
|
||||
\"namespace\": \"${CI_ENVIRONMENT_NAME}\"
|
||||
}
|
||||
}
|
||||
}" \
|
||||
your-charybdis-host:50051 \
|
||||
charybdis.entities.EntityService/CreateEntity
|
||||
```
|
||||
|
||||
## Enabling Security
|
||||
|
||||
For production use, enable mTLS and RBAC:
|
||||
|
||||
### Step 1: Generate Certificates
|
||||
|
||||
```bash
|
||||
# Use the provided test script
|
||||
./test-mtls-rbac.sh
|
||||
```
|
||||
|
||||
This creates:
|
||||
- `certs/ca.pem` - Certificate Authority
|
||||
- `certs/server-cert.pem` / `server-key.pem` - Server certificate
|
||||
- `certs/admin-cert.pem` / `admin-key.pem` - Admin client certificate
|
||||
|
||||
### Step 2: Enable Security
|
||||
|
||||
```bash
|
||||
export SECURITY_MTLS_ENABLED=true
|
||||
export SECURITY_MTLS_SERVER_CERT=./certs/server-cert.pem
|
||||
export SECURITY_MTLS_SERVER_KEY=./certs/server-key.pem
|
||||
export SECURITY_MTLS_CLIENT_CA=./certs/ca.pem
|
||||
export SECURITY_RBAC_ENABLED=true
|
||||
```
|
||||
|
||||
### Step 3: Test with mTLS
|
||||
|
||||
```bash
|
||||
grpcurl \
|
||||
-cacert certs/ca.pem \
|
||||
-cert certs/admin-cert.pem \
|
||||
-key certs/admin-key.pem \
|
||||
-d '{}' \
|
||||
localhost:50051 charybdis.entities.EntityService/ListEntities
|
||||
```
|
||||
|
||||
See the [Security Guide](security.md) for detailed configuration.
|
||||
|
||||
## Backstage Migration (Optional)
|
||||
|
||||
If you currently use Backstage and want to migrate gradually, the built-in YAML adapter serves entities in Backstage format:
|
||||
|
||||
```yaml
|
||||
# backstage app-config.yaml
|
||||
catalog:
|
||||
locations:
|
||||
- type: url
|
||||
target: http://your-charybdis-host:8080/yaml/locations
|
||||
rules:
|
||||
- allow: [Component, System, Service]
|
||||
```
|
||||
|
||||
Backstage will automatically discover and import entities from Charybdis. You can run both in parallel — entities managed via gRPC are immediately visible in Backstage.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Port Already in Use
|
||||
|
||||
```
|
||||
Error: transport error
|
||||
```
|
||||
|
||||
**Solution**: Check if another process is using port 50051:
|
||||
|
||||
```bash
|
||||
lsof -ti:50051
|
||||
```
|
||||
|
||||
Kill the process or change the port:
|
||||
|
||||
```bash
|
||||
export GRPC_PORT=50052
|
||||
```
|
||||
|
||||
### Database Connection Failed
|
||||
|
||||
```
|
||||
Error: password authentication failed
|
||||
```
|
||||
|
||||
**Solution**: Verify your DATABASE_URL:
|
||||
|
||||
```bash
|
||||
# Test connection
|
||||
psql "$DATABASE_URL" -c "SELECT 1;"
|
||||
```
|
||||
|
||||
### Permission Denied (with security enabled)
|
||||
|
||||
```
|
||||
Code: PermissionDenied
|
||||
Message: Role 'X' does not have permission 'Y'
|
||||
```
|
||||
|
||||
**Solution**: Check your certificate and role mappings. See [Security Guide](security.md).
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that you have Charybdis running:
|
||||
|
||||
1. Learn about [Core Concepts](core-concepts.md) — entities, vulnerabilities, events
|
||||
2. Read the [Vision & Roadmap](../VISION.md) — where Charybdis is going
|
||||
3. Configure [Security](security.md) for production (mTLS + RBAC)
|
||||
4. Explore [Plugins](../plugins/README.md) for external integrations
|
||||
|
||||
## Configuration
|
||||
|
||||
Charybdis supports two configuration methods:
|
||||
|
||||
### 1. Configuration File (Recommended)
|
||||
|
||||
Use `config.toml` for structured configuration:
|
||||
|
||||
```toml
|
||||
# config.toml
|
||||
[server]
|
||||
grpc_host = "[::1]"
|
||||
grpc_port = 50051
|
||||
|
||||
[database]
|
||||
url = "${DATABASE_URL}" # Environment variable substitution
|
||||
|
||||
[security.mtls]
|
||||
enabled = false
|
||||
|
||||
[security.rbac]
|
||||
enabled = false
|
||||
|
||||
[telemetry]
|
||||
service_name = "charybdis"
|
||||
environment = "development"
|
||||
enable_console = true
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- ✅ Organized by section (server, database, security, telemetry, plugins)
|
||||
- ✅ Environment variable substitution with `${VAR_NAME}`
|
||||
- ✅ Comments and documentation inline
|
||||
- ✅ Easy to version control (excluding secrets)
|
||||
- ✅ No need to export dozens of environment variables
|
||||
|
||||
**Using Environment Variables in config.toml:**
|
||||
|
||||
```toml
|
||||
[database]
|
||||
url = "${DATABASE_URL}" # Will be substituted at runtime
|
||||
|
||||
[plugins.defectdojo]
|
||||
api_key = "${DEFECTDOJO_API_KEY}" # Secrets stay in environment
|
||||
```
|
||||
|
||||
Then set only the secrets:
|
||||
|
||||
```bash
|
||||
export DATABASE_URL="postgresql://..."
|
||||
export DEFECTDOJO_API_KEY="secret-key"
|
||||
```
|
||||
|
||||
### 2. Environment Variables (Legacy)
|
||||
|
||||
If `config.toml` is not found, Charybdis falls back to environment variables:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `DATABASE_URL` | (required) | PostgreSQL connection string |
|
||||
| `GRPC_HOST` | `[::1]` | gRPC server bind address |
|
||||
| `GRPC_PORT` | `50051` | gRPC server port |
|
||||
| `RUST_LOG` | `info` | Logging level |
|
||||
| `SECURITY_MTLS_ENABLED` | `false` | Enable mTLS authentication |
|
||||
| `SECURITY_RBAC_ENABLED` | `false` | Enable RBAC authorization |
|
||||
| `OTEL_ENABLE_CONSOLE` | `true` | Enable console logging |
|
||||
| `OTEL_SERVICE_NAME` | `charybdis` | Service name for telemetry |
|
||||
|
||||
### Configuration File Locations
|
||||
|
||||
Charybdis looks for configuration files in this order:
|
||||
|
||||
1. `./config.toml` (current directory)
|
||||
2. `./charybdis.toml`
|
||||
3. `/etc/charybdis/config.toml` (Linux/Unix)
|
||||
|
||||
If none are found, it uses environment variables.
|
||||
|
||||
See `config.toml.example` for a complete configuration template with all options documented.
|
||||
|
||||
---
|
||||
|
||||
**Need help?** Check the [troubleshooting guide](troubleshooting.md) or [open an issue](../../issues).
|
||||
Reference in New Issue
Block a user