ComplyTime logo ComplyTime Get Started

Development

This guide provides comprehensive instructions for setting up, building, and testing the ComplyBeacon project. It complements the DESIGN.md document by focusing on the practical aspects of development.

Prerequisites#

Required Software#

  • Go 1.25+: The project uses Go 1.25.8 with toolchain 1.25.9
  • Podman: For containerized development and deployment (Docker is not supported)
  • Task: For build automation (installation guide)
  • Git: For version control
  • openssl: Cryptography toolkit

Development Environment Setup#

1. Clone The Repository#

git clone https://github.com/complytime/complybeacon.git
cd complybeacon

2. Install Task (if Needed)#

The project uses Task for build automation. Install it if you don’t have it:

## Macos
brew install go-task/tap/go-task

## Linux
sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b ~/.local/bin

## Or Using Go
go install github.com/go-task/task/v3/cmd/task@latest

## Verify Installation
task --version

3. Initialize Go Workspace#

The project uses Go workspaces to manage multiple modules:

task workspace

This creates a go.work file that includes all project modules:

  • ./proofwatch
  • ./truthbeam

4. Install Dependencies#

Dependencies are managed per module. Install them for all modules:

task deps

This automatically:

  • Syncs the Go workspace
  • Runs go mod tidy for each module
  • Verifies and downloads dependencies

5. Verify Installation#

## Run Tests To Verify Everything Works
task test

## Run All Quality Gates (lint + Test)
task check

Project Structure#

complybeacon/
├── compose.yaml                # Container orchestration configuration
├── Taskfile.yml                # Build automation
├── .taskfiles/                 # Task modules and helper scripts
├── docs/                       # Documentation
│   ├── DESIGN.md              # Architecture and design documentation
│   ├── DEVELOPMENT.md         # This file
│   └── attributes/            # Attribute documentation
├── model/                      # OpenTelemetry semantic conventions
│   ├── attributes.yaml        # Attribute definitions
│   └── entities.yaml          # Entity definitions
├── proofwatch/                 # ProofWatch instrumentation library
│   ├── attributes.go          # Attribute definitions
│   ├── evidence.go            # Evidence types
│   └── proofwatch.go          # Main library
├── truthbeam/                  # TruthBeam processor module
│   ├── internal/              # Internal packages
│   ├── config.go              # Configuration
│   └── processor.go           # Main processor logic
├── beacon-distro/              # OpenTelemetry Collector distribution
│   ├── config.yaml            # Collector configuration
│   └── Containerfile.collector # Container definition
├── hack/                       # Development utilities
│   ├── demo/                  # Demo configurations
│   ├── sampledata/            # Sample data for testing
└── bin/                        # Built binaries (created by task infra:deploy)

Testing#

Running Tests#

## Run All Tests (includes Version Checks And Coverage)
task test

## Run Tests With Race Detection
task test-race

## Generate Coverage Reports
task dev:coverage-report

## Run Tests For Specific Module
cd proofwatch && go test -v ./...
cd truthbeam && go test -v ./...

Integration Testing#

The project includes integration tests using the demo environment:

## Start The Demo Environment (builds Images And Starts Services)
task deploy

## Or Run In Background
podman-compose -f compose.yaml up -d

## Test The Pipeline
curl -X POST http://localhost:8088/eventsource/receiver \
  -H "Content-Type: application/json" \
  -d @hack/sampledata/evidence.json

## View Logs
podman-compose -f compose.yaml logs -f

## Stop The Environment
task infra:undeploy

## Check Logs In Grafana At Http://localhost:3000

Component Development#

1. Proofwatch Development#

ProofWatch is an instrumentation library for emitting compliance evidence.

Key Files:

  • proofwatch/proofwatch.go - Main library interface
  • proofwatch/evidence.go - Evidence type definition
  • proofwatch/attributes.go - OpenTelemetry attributes

Development Workflow:

cd proofwatch

## Run Tests
go test -v ./...

## Run Linting (from Root)
cd ..
task lint

## Format Code
go fmt ./...

2. Compass Development#

Compass is an external enrichment service that TruthBeam connects to for compliance lookups. It must be provided separately and is not included in the demo stack.

3. Truthbeam Development#

TruthBeam is an OpenTelemetry Collector processor for enriching logs.

Key Files:

  • truthbeam/processor.go - Main processor logic
  • truthbeam/config.go - Configuration structures
  • truthbeam/factory.go - Processor factory

Development Workflow:

cd truthbeam

## Run Tests
go test -v ./...

## Test With Collector (requires Beacon-distro)
cd ../beacon-distro
## Modify Config To Use Local Truthbeam
## Run Collector With Local Processor

Local development config

If you want locally test the TruthBeam, remember to change the manifest.yaml

Add replace directive at the end of manifest.yaml, to make sure collector use your truthbeam code. Default collector will use - gomod: github.com/complytime/complybeacon/truthbeam main

For example:

replaces:
  - github.com/complytime/complybeacon/truthbeam => github.com/AlexXuan233/complybeacon/truthbeam 52e4a76ea0f72a7049e73e7a5d67d988116a3892

or

replaces:
  - github.com/complytime/complybeacon/truthbeam => github.com/AlexXuan233/complybeacon/truthbeam main

4. Beacon Distro Development#

The Beacon distribution is a custom OpenTelemetry Collector.

Key Files:

  • beacon-distro/config.yaml - Collector configuration
  • beacon-distro/Containerfile.collector - Container definition

Development Workflow:

## Build The Collector Image
podman build -f beacon-distro/Containerfile.collector -t complybeacon-collector beacon-distro/

## Or Force Rebuild Without Cache
podman build --no-cache -f beacon-distro/Containerfile.collector -t complybeacon-collector beacon-distro/

## Run Locally For Quick Testing
podman run --rm complybeacon-collector --config /etc/otelcol-beacon/config.yaml

## Full Stack Deployment For Integration Testing
task deploy

Debugging And Troubleshooting#

Debugging Tools#

## View All Container Logs
podman-compose -f compose.yaml logs -f

## View Specific Service Logs
podman-compose -f compose.yaml ps            # List running services
podman-compose -f compose.yaml logs -f collector

## Check Container Status
podman images | grep complybeacon            # List built images
podman inspect complybeacon-collector        # Inspect image details

Code Generation#

The project uses several code generation tools:

1. Opentelemetry Semantic Conventions#

Generate documentation and Go code from semantic convention models:

## Generate Documentation
task codegen:weaver-docsgen

## Generate Go Code
task codegen:weaver-codegen

## Validate Models
task codegen:weaver-check

## Validate Logs Against Semantic Conventions
task codegen:weaver-semantic-check

2. Manual Code Generation#

If you modify the semantic conventions:

## Update Semantic Conventions
vim model/attributes.yaml
vim model/entities.yaml

## Regenerate All Code (api + Weaver)
task codegen:api-codegen
task codegen:weaver-codegen

Deployment And Demo#

Local Development Demo#

The demo environment orchestrates multiple containers (Grafana, Loki, Beacon Collector, Compass).

  1. Start the full stack:
## Interactive Mode (shows Logs In Terminal)
task infra:deploy

## Or Background/detached Mode
podman-compose -f compose.yaml up -d

This automatically:

  • Syncs OTel versions from truthbeam to beacon-distro
  • Builds the beacon collector image
  • Starts all services (Grafana, Loki, Collector)
  1. Test the pipeline:
curl -X POST http://localhost:8088/eventsource/receiver \
  -H "Content-Type: application/json" \
  -d @hack/sampledata/evidence.json
  1. View results:
  1. Stop the stack:
task infra:undeploy

Additional Resources#

For questions or support, please open an issue in the GitHub repository.

Edit this page on GitHub
Prev
Design
Next
Policy